InnoDB Clusterは、MySQLの高可用性構成をサポートするMySQLの可用性フレームワークです。
まだLab版ですが、既にRCになっているMySQL Group Replication、GAになっているMySQL Router (GAはバージョンは1.x),MySQL Shell (DMR)を組み合わせて高可用性を実現しています。
基本的な動作は、MySQL Fabricに似ていますが、管理リポジトリー自体がGroup Replication内にあるので、デフォルトで冗長化されています。また、MySQL Router,MySQL Shell,MySQL Group Replicationが蜜に連携していて1つのHAパッケージのようになっています。まだ、十分に検証出来ていませんが、基本的な動作を確認したので、此方にメモしておきます。
※まだLab版なので、仕様は変わると思います。

innodb_cluster

ダウンロード
MySQL Labs :: MySQL InnoDB Cluster 5.7.15 Preview
※ インストール方法は、RPMが提供されているので割愛しています。

■ Sandboxインスタンスのデプロイ
※ Do not run MySQL Shell as root.
以下のdeployLocalInstanceコマンドで新規インスタンスが構築されます。


[root@replications admin]# su - mysql
最終ログイン: 2016/09/27 (火) 22:32:27 JST日時 pts/2
-bash-4.2$ mysqlsh
Welcome to MySQL Shell 1.0.5-labs Development Preview

Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type '\help', '\h' or '\?' for help, type '\quit' or '\q' to exit.

Currently in JavaScript mode. Use \sql to switch to SQL mode and execute queries.
mysql-js> dba.deployLocalInstance(3310)
A new MySQL sandbox instance will be created on this host in 
/home/mysql/mysql-sandboxes/3310

Please enter a MySQL root password for the new instance: 
Deploying new MySQL instance...

Instance localhost:3310 successfully deployed and started.
Use '\connect root@localhost:3310' to connect to the instance.

mysql-js> dba.deployLocalInstance(3320) 
A new MySQL sandbox instance will be created on this host in 
/home/mysql/mysql-sandboxes/3320

Please enter a MySQL root password for the new instance: 
Deploying new MySQL instance...

Instance localhost:3320 successfully deployed and started.
Use '\connect root@localhost:3320' to connect to the instance.

mysql-js> dba.deployLocalInstance(3330)
A new MySQL sandbox instance will be created on this host in 
/home/mysql/mysql-sandboxes/3330

Please enter a MySQL root password for the new instance: 
Deploying new MySQL instance...

Instance localhost:3330 successfully deployed and started.
Use '\connect root@localhost:3330' to connect to the instance.

mysql-js> 


[root@replications mysql-sandboxes]# pwd
/home/mysql/mysql-sandboxes
[root@replications mysql-sandboxes]# ls -l
合計 12
drwxrwxr-x. 4 mysql mysql 4096  9月 27 17:25 3310
drwxrwxr-x. 4 mysql mysql 4096  9月 27 17:26 3320
drwxrwxr-x. 4 mysql mysql 4096  9月 27 17:27 3330
[root@replications mysql-sandboxes]# ls -l 3310/
合計 28
-rw-r-----. 1 mysql mysql    5  9月 27 17:25 3310.pid
drwxr-x---. 5 mysql mysql 4096  9月 27 17:25 data
-rw-------. 1 mysql mysql  806  9月 27 17:25 my.cnf
drwxrwxr-x. 2 mysql mysql    6  9月 27 17:25 mysql-files
srwxrwxrwx. 1 mysql mysql    0  9月 27 17:25 mysqld.sock
-rw-------. 1 mysql mysql    5  9月 27 17:25 mysqld.sock.lock
srwxrwxrwx. 1 mysql mysql    0  9月 27 17:25 mysqlx.sock
-rw-------. 1 mysql mysql    6  9月 27 17:25 mysqlx.sock.lock
-rwx------. 1 mysql mysql  121  9月 27 17:25 start.sh
-rwx------. 1 mysql mysql  186  9月 27 17:25 stop.sh
[root@replications mysql-sandboxes]# ls -l 3310/data/
合計 122948
-rw-r-----. 1 mysql mysql      169  9月 27 17:25 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:25 ON.000002
-rw-r-----. 1 mysql mysql      150  9月 27 17:25 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:25 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:25 auto.cnf
-rw-r-----. 1 mysql mysql    14266  9月 27 17:25 error.log
-rw-r-----. 1 mysql mysql      362  9月 27 17:25 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:25 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:25 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:25 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:25 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:25 mysql
drwxr-x---. 2 mysql mysql     8192  9月 27 17:25 performance_schema
drwxr-x---. 2 mysql mysql     8192  9月 27 17:25 sys
[root@replications mysql-sandboxes]# ls -l 3320/data/
合計 122948
-rw-r-----. 1 mysql mysql      169  9月 27 17:26 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:26 ON.000002
-rw-r-----. 1 mysql mysql      150  9月 27 17:26 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:26 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:26 auto.cnf
-rw-r-----. 1 mysql mysql    14369  9月 27 17:26 error.log
-rw-r-----. 1 mysql mysql      362  9月 27 17:26 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:26 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:26 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:26 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:26 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:26 mysql
drwxr-x---. 2 mysql mysql     8192  9月 27 17:26 performance_schema
drwxr-x---. 2 mysql mysql     8192  9月 27 17:26 sys
[root@replications mysql-sandboxes]# ls -l 3330/data/
合計 122948
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000002
-rw-r-----. 1 mysql mysql      150  9月 27 17:27 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:27 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:27 auto.cnf
-rw-r-----. 1 mysql mysql    13201  9月 27 17:27 error.log
-rw-r-----. 1 mysql mysql      359  9月 27 17:27 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:27 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:27 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:27 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:27 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:27 mysql
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 performance_schema
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 sys
[root@replications mysql-sandboxes]# ls -l 3330/data/
合計 122948
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000002
-rw-r-----. 1 mysql mysql      150  9月 27 17:27 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:27 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:27 auto.cnf
-rw-r-----. 1 mysql mysql    13201  9月 27 17:27 error.log
-rw-r-----. 1 mysql mysql      359  9月 27 17:27 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:27 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:27 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:27 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:27 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:27 mysql
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 performance_schema
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 sys
[root@replications mysql-sandboxes]# 

■ InnoDB Clusterの初期化
※Do not lose the MASTER key because it is required for managing the InnoDB cluster.
構築したインスタンスでグループレプリケーションを構築します。


mysql-js> \connect root@localhost:3310
Creating a Session to 'root@localhost:3310'
Enter password: 
Classic Session successfully established. No default schema selected.
mysql-js> cluster = dba.createCluster('test')
A new InnoDB cluster will be created on instance 'root@localhost:3310'.

When setting up a new InnoDB cluster it is required to define an administrative
MASTER key for the cluster. This MASTER key needs to be re-entered when making
changes to the cluster later on, e.g.adding new MySQL instances or configuring
MySQL Routers. Losing this MASTER key will require the configuration of all
InnoDB cluster entities to be changed.

Please specify an administrative MASTER key for the cluster 'test': 
Creating InnoDB cluster 'test' on 'root@localhost:3310'...
Adding Seed Instance...

Cluster successfully created. Use Cluster.addInstance() to add MySQL instances.
At least 3 instances are needed for the cluster to be able to withstand up to
one server failure.

<Cluster:test>
mysql-js> 



[root@replications mysql-sandboxes]# ls -l 3310/data/
合計 122988
-rw-r-----. 1 mysql mysql      169  9月 27 17:25 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:25 ON.000002
-rw-r-----. 1 mysql mysql    10397  9月 27 17:31 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:25 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:25 auto.cnf
-rw-r-----. 1 mysql mysql    21676  9月 27 17:31 error.log
-rw-r-----. 1 mysql mysql      362  9月 27 17:25 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:31 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:25 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:31 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:31 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:25 mysql
drwxr-x---. 2 mysql mysql     4096  9月 27 17:31 mysql_innodb_cluster_metadata
drwxr-x---. 2 mysql mysql     8192  9月 27 17:25 performance_schema
-rw-r-----. 1 mysql mysql      232  9月 27 17:31 replications-relay-bin-group_replication_applier.000001
-rw-r-----. 1 mysql mysql      598  9月 27 17:31 replications-relay-bin-group_replication_applier.000002
-rw-r-----. 1 mysql mysql      116  9月 27 17:31 replications-relay-bin-group_replication_applier.index
-rw-r-----. 1 mysql mysql      150  9月 27 17:31 replications-relay-bin-group_replication_recovery.000001
-rw-r-----. 1 mysql mysql       59  9月 27 17:31 replications-relay-bin-group_replication_recovery.index
drwxr-x---. 2 mysql mysql     8192  9月 27 17:25 sys
[root@replications mysql-sandboxes]# ls -l 3320/data/
合計 122948
-rw-r-----. 1 mysql mysql      169  9月 27 17:26 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:26 ON.000002
-rw-r-----. 1 mysql mysql      150  9月 27 17:26 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:26 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:26 auto.cnf
-rw-r-----. 1 mysql mysql    14369  9月 27 17:26 error.log
-rw-r-----. 1 mysql mysql      362  9月 27 17:26 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:26 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:26 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:26 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:26 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:26 mysql
drwxr-x---. 2 mysql mysql     8192  9月 27 17:26 performance_schema
drwxr-x---. 2 mysql mysql     8192  9月 27 17:26 sys
[root@replications mysql-sandboxes]# ls -l 3330/data/
合計 122948
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000002
-rw-r-----. 1 mysql mysql      150  9月 27 17:27 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:27 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:27 auto.cnf
-rw-r-----. 1 mysql mysql    13201  9月 27 17:27 error.log
-rw-r-----. 1 mysql mysql      359  9月 27 17:27 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:27 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:27 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:27 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:27 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:27 mysql
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 performance_schema
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 sys
[root@replications mysql-sandboxes]# 

■ InnoDB Clusterへのインスタンスの追加
グループレプリケーションへインスタンスを追加します。


mysql-js> cluster.addInstance('root@localhost:3320')
A new instance will be added to the InnoDB cluster. Depending on the amount of
data on the cluster this might take from a few seconds to several hours.

Please provide the password for 'root@localhost:3320': 
Adding instance to the cluster ...

The instance 'root@localhost:3320' was successfully added to the cluster.

mysql-js> cluster.addInstance('root@localhost:3330')
A new instance will be added to the InnoDB cluster. Depending on the amount of
data on the cluster this might take from a few seconds to several hours.

Please provide the password for 'root@localhost:3330': 
Adding instance to the cluster ...

The instance 'root@localhost:3330' was successfully added to the cluster.

mysql-js> 


[root@replications mysql-sandboxes]# ls -l 3310/data/
合計 122988
-rw-r-----. 1 mysql mysql      169  9月 27 17:25 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:25 ON.000002
-rw-r-----. 1 mysql mysql    12253  9月 27 17:36 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:25 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:25 auto.cnf
-rw-r-----. 1 mysql mysql    23760  9月 27 17:36 error.log
-rw-r-----. 1 mysql mysql      362  9月 27 17:25 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:36 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:25 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:36 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:36 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:25 mysql
drwxr-x---. 2 mysql mysql     4096  9月 27 17:31 mysql_innodb_cluster_metadata
drwxr-x---. 2 mysql mysql     8192  9月 27 17:25 performance_schema
-rw-r-----. 1 mysql mysql      232  9月 27 17:31 replications-relay-bin-group_replication_applier.000001
-rw-r-----. 1 mysql mysql     1336  9月 27 17:36 replications-relay-bin-group_replication_applier.000002
-rw-r-----. 1 mysql mysql      116  9月 27 17:31 replications-relay-bin-group_replication_applier.index
-rw-r-----. 1 mysql mysql      150  9月 27 17:31 replications-relay-bin-group_replication_recovery.000001
-rw-r-----. 1 mysql mysql       59  9月 27 17:31 replications-relay-bin-group_replication_recovery.index
drwxr-x---. 2 mysql mysql     8192  9月 27 17:25 sys
[root@replications mysql-sandboxes]# ls -l 3320/data/
合計 122996
-rw-r-----. 1 mysql mysql      169  9月 27 17:26 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:26 ON.000002
-rw-r-----. 1 mysql mysql    12129  9月 27 17:36 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:26 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:26 auto.cnf
-rw-r-----. 1 mysql mysql    24847  9月 27 17:37 error.log
-rw-r-----. 1 mysql mysql      362  9月 27 17:26 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:36 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:26 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:36 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:35 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:26 mysql
drwxr-x---. 2 mysql mysql     4096  9月 27 17:35 mysql_innodb_cluster_metadata
drwxr-x---. 2 mysql mysql     8192  9月 27 17:26 performance_schema
-rw-r-----. 1 mysql mysql      232  9月 27 17:35 replications-relay-bin-group_replication_applier.000001
-rw-r-----. 1 mysql mysql     2135  9月 27 17:36 replications-relay-bin-group_replication_applier.000002
-rw-r-----. 1 mysql mysql      116  9月 27 17:35 replications-relay-bin-group_replication_applier.index
-rw-r-----. 1 mysql mysql      233  9月 27 17:35 replications-relay-bin-group_replication_recovery.000001
-rw-r-----. 1 mysql mysql      269  9月 27 17:35 replications-relay-bin-group_replication_recovery.000002
-rw-r-----. 1 mysql mysql      118  9月 27 17:35 replications-relay-bin-group_replication_recovery.index
drwxr-x---. 2 mysql mysql     8192  9月 27 17:26 sys
[root@replications mysql-sandboxes]# ls -l 3330/data/
合計 122992
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000001
-rw-r-----. 1 mysql mysql      169  9月 27 17:27 ON.000002
-rw-r-----. 1 mysql mysql    12129  9月 27 17:36 ON.000003
-rw-r-----. 1 mysql mysql       36  9月 27 17:27 ON.index
-rw-r-----. 1 mysql mysql       56  9月 27 17:27 auto.cnf
-rw-r-----. 1 mysql mysql    22841  9月 27 17:36 error.log
-rw-r-----. 1 mysql mysql      359  9月 27 17:27 ib_buffer_pool
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:36 ib_logfile0
-rw-r-----. 1 mysql mysql 50331648  9月 27 17:27 ib_logfile1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:36 ibdata1
-rw-r-----. 1 mysql mysql 12582912  9月 27 17:36 ibtmp1
drwxr-x---. 2 mysql mysql     4096  9月 27 17:27 mysql
drwxr-x---. 2 mysql mysql     4096  9月 27 17:36 mysql_innodb_cluster_metadata
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 performance_schema
-rw-r-----. 1 mysql mysql      232  9月 27 17:36 replications-relay-bin-group_replication_applier.000001
-rw-r-----. 1 mysql mysql     1202  9月 27 17:36 replications-relay-bin-group_replication_applier.000002
-rw-r-----. 1 mysql mysql      116  9月 27 17:36 replications-relay-bin-group_replication_applier.index
-rw-r-----. 1 mysql mysql      233  9月 27 17:36 replications-relay-bin-group_replication_recovery.000001
-rw-r-----. 1 mysql mysql      269  9月 27 17:36 replications-relay-bin-group_replication_recovery.000002
-rw-r-----. 1 mysql mysql      118  9月 27 17:36 replications-relay-bin-group_replication_recovery.index
drwxr-x---. 2 mysql mysql     8192  9月 27 17:27 sys
[root@replications mysql-sandboxes]# 


[root@replications mysql-sandboxes]# netstat -na | grep sandboxes
unix  2      [ ACC ]     STREAM     LISTENING     26245    /home/mysql/mysql-sandboxes/3310/mysqlx.sock
unix  2      [ ACC ]     STREAM     LISTENING     26250    /home/mysql/mysql-sandboxes/3310/mysqld.sock
unix  2      [ ACC ]     STREAM     LISTENING     22039    /home/mysql/mysql-sandboxes/3330/mysqlx.sock
unix  2      [ ACC ]     STREAM     LISTENING     22044    /home/mysql/mysql-sandboxes/3330/mysqld.sock
unix  2      [ ACC ]     STREAM     LISTENING     21930    /home/mysql/mysql-sandboxes/3320/mysqlx.sock
unix  2      [ ACC ]     STREAM     LISTENING     21935    /home/mysql/mysql-sandboxes/3320/mysqld.sock
[root@replications mysql-sandboxes]# 

■ InnoDB Clusterのステータス確認
全てのホストがONLINEでHAが構成されている事が確認出来る。


mysql-js> cluster.status()
{
    "clusterName": "test",
    "defaultReplicaSet": {
        "status": "Cluster tolerant to up to ONE failure.",
        "topology": {
            "localhost:3310": {
                "address": "localhost:3310",
                "status": "ONLINE",
                "role": "HA",
                "mode": "R/W",
                "leaves": {
                    "localhost:3320": {
                        "address": "localhost:3320",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    },
                    "localhost:3330": {
                        "address": "localhost:3330",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    }
                }
            }
        }
    }
}
mysql-js> 

Sandbox環境のGroup ReplicationはSingle Primary Modeで動いています。
It is a configuration mode for Group Replication that makes a single member act as a writeable master (PRIMARY) and the rest of the members act as hot-standbys (SECONDARY).
The group itself coordinates and configures itself automatically to determine which member will act as the PRIMARY, through a leader election mechanism.


-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6446 -e "show variables like 'group_replication_single_primary_mode'"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---------------------------------------+-------+
| Variable_name                         | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON    |
+---------------------------------------+-------+
-bash-4.2$ 

-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6446 -e "show variables like 'group_replication_enforce_update_everywhere_checks'"
mysql: [Warning] Using a password on the command line interface can be insecure.
+----------------------------------------------------+-------+
| Variable_name                                      | Value |
+----------------------------------------------------+-------+
| group_replication_enforce_update_everywhere_checks | OFF   |
+----------------------------------------------------+-------+
-bash-4.2$ 

-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6447 -e "SELECT * FROM performance_schema.global_status WHERE VARIABLE_NAME='group_replication_primary_member'"
mysql: [Warning] Using a password on the command line interface can be insecure.
+----------------------------------+--------------------------------------+
| VARIABLE_NAME                    | VARIABLE_VALUE                       |
+----------------------------------+--------------------------------------+
| group_replication_primary_member | 944915c6-853a-11e6-8e3d-080027d65c57 |
+----------------------------------+--------------------------------------+
-bash-4.2$ 

■ MySQL Routerのデプロイ
MySQL Routerを起動させて、グループレプリケーションへの接続をコントロールします。


-bash-4.2$ sudo mysqlrouter --bootstrap localhost:3310
[sudo] password for mysql: 
Please enter the administrative MASTER key for the MySQL InnoDB cluster: 
MySQL Router has now been configured for the InnoDB cluster 'test'.

The following connection information can be used to connect to the cluster.

Classic MySQL protocol connections to cluster 'test':
- Read/Write Connections: localhost:6446
- Read/Only Connections: localhost:6447
-bash-4.2$ 


-bash-4.2$ mysqlrouter &
[1] 3615
-bash-4.2$ Logging to /var/log/mysqlrouter/mysqlrouter.log

-bash-4.2$ ps -ef | grep mysqlrouter
mysql     3615  3589  0 14:32 pts/1    00:00:00 mysqlrouter
mysql     3625  3589  0 14:32 pts/1    00:00:00 grep --color=auto mysqlrouter
-bash-4.2$ 

■ MySQLRouter経由でmysqlに接続し動作確認
mysqlshell経由


1) MySQL Shell経由

-bash-4.2$ mysqlsh --uri root@localhost:6446
Creating a Session to 'root@localhost:6446'
Enter password: 
Classic Session successfully established. No default schema selected.
Welcome to MySQL Shell 1.0.5-labs Development Preview

Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type '\help', '\h' or '\?' for help, type '\quit' or '\q' to exit.

Currently in JavaScript mode. Use \sql to switch to SQL mode and execute queries.
mysql-js> \sql
Switching to SQL mode... Commands end with ;
mysql-sql> select @@hostname,@@port;
+--------------+--------+
| @@hostname   | @@port |
+--------------+--------+
| replications |   3310 |
+--------------+--------+
1 row in set (0.00 sec)
mysql-sql> 


-bash-4.2$ mysqlsh --uri root@localhost:6447
Creating a Session to 'root@localhost:6447'
Enter password: 
Classic Session successfully established. No default schema selected.
Welcome to MySQL Shell 1.0.5-labs Development Preview

Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type '\help', '\h' or '\?' for help, type '\quit' or '\q' to exit.

Currently in JavaScript mode. Use \sql to switch to SQL mode and execute queries.
mysql-js> \sql
Switching to SQL mode... Commands end with ;
mysql-sql> select @@hostname,@@port;
+--------------+--------+
| @@hostname   | @@port |
+--------------+--------+
| replications |   3330 |
+--------------+--------+
1 row in set (0.00 sec)
mysql-sql>

2) MySQLコマンド経由での確認とステータスの確認


-bash-4.2$ mysql -u root -p -h 127.0.0.1 -P 6446
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 42
Server version: 5.7.15-labs-gr090-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select @@hostname,@@port;
+--------------+--------+
| @@hostname   | @@port |
+--------------+--------+
| replications |   3310 |
+--------------+--------+
1 row in set (0.00 sec)

mysql> 


-bash-4.2$ mysql -u root -p -h 127.0.0.1 -P 6447
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 5.7.15-labs-gr090-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select @@hostname,@@port;
+--------------+--------+
| @@hostname   | @@port |
+--------------+--------+
| replications |   3320 |
+--------------+--------+
1 row in set (0.00 sec)

mysql> 



mysql-js> cluster.status()
{
    "clusterName": "test",
    "defaultReplicaSet": {
        "status": "Cluster tolerant to up to ONE failure.",
        "topology": {
            "localhost:3310": {
                "address": "localhost:3310",
                "status": "ONLINE",
                "role": "HA",
                "mode": "R/W",
                "leaves": {
                    "localhost:3320": {
                        "address": "localhost:3320",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    },
                    "localhost:3330": {
                        "address": "localhost:3330",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    }
                }
            }
        }
    }
}
mysql-js> 

mysql-js> cluster.describe();
{
    "clusterName": "test",
    "adminType": "local",
    "defaultReplicaSet": {
        "name": "default",
        "instances": [
            {
                "name": "localhost:3310",
                "host": "localhost:3310",
                "role": "HA"
            },
            {
                "name": "localhost:3320",
                "host": "localhost:3320",
                "role": "HA"
            },
            {
                "name": "localhost:3330",
                "host": "localhost:3330",
                "role": "HA"
            }
        ]
    }
}
mysql-js> 

■ フェイルオーバーの確認 (HA)
既存のマスターインスタンスをdba.killLocalInstanceでダウンさせてみる。


mysql-js> dba.killLocalInstance(3310)
The MySQL sandbox instance on this host in 
/home/mysql/mysql-sandboxes/3310 will be killed

Killing MySQL instance...

Instance localhost:3310 successfully killed.

mysql-js> 

– 書き込みの接続が3310から3320に代わっている事が確認出来ます。


-bash-4.2$ mysqlsh --uri root@localhost:6446
Creating a Session to 'root@localhost:6446'
Enter password: 
ERROR: 2003 (HY000): Can't connect to remote MySQL server on '127.0.0.1:6446'
-bash-4.2$ mysqlsh --uri root@localhost:6446
Creating a Session to 'root@localhost:6446'
Enter password: 
Classic Session successfully established. No default schema selected.
Welcome to MySQL Shell 1.0.5-labs Development Preview

Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type '\help', '\h' or '\?' for help, type '\quit' or '\q' to exit.

Currently in JavaScript mode. Use \sql to switch to SQL mode and execute queries.
mysql-js> \sql
Switching to SQL mode... Commands end with ;
mysql-sql> select @@hostname,@@port;
+--------------+--------+
| @@hostname   | @@port |
+--------------+--------+
| replications |   3320 |
+--------------+--------+
1 row in set (0.00 sec)
mysql-sql> 



-bash-4.2$ mysql -u root -p -h 127.0.0.1 -P 6446
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 35
Server version: 5.7.15-labs-gr090-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select @@hostname,@@port;
+--------------+--------+
| @@hostname   | @@port |
+--------------+--------+
| replications |   3320 |
+--------------+--------+
1 row in set (0.00 sec)

mysql> 


-bash-4.2$ mysql -u root -p -h 127.0.0.1 -P 6447
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 24
Server version: 5.7.15-labs-gr090-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select @@hostname,@@port;
+--------------+--------+
| @@hostname   | @@port |
+--------------+--------+
| replications |   3330 |
+--------------+--------+
1 row in set (0.00 sec)

mysql> 


-bash-4.2$ mysqlsh
Welcome to MySQL Shell 1.0.5-labs Development Preview

Copyright (c) 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type '\help', '\h' or '\?' for help, type '\quit' or '\q' to exit.

Currently in JavaScript mode. Use \sql to switch to SQL mode and execute queries.
mysql-js> \connect root@localhost:6446
Creating a Session to 'root@localhost:6446'
Enter password: 
Classic Session successfully established. No default schema selected.
mysql-js> cluster = dba.getCluster()
When the InnoDB cluster was setup, a MASTER key was defined in order to enable
performing administrative tasks on the cluster.

Please specify the administrative MASTER key for the default cluster: 
<Cluster:test>
mysql-js> cluster.status()
{
    "clusterName": "test",
    "defaultReplicaSet": {
        "status": "Cluster is NOT tolerant to any failures.",
        "topology": {
            "localhost:3320": {
                "address": "localhost:3320",
                "status": "ONLINE",
                "role": "HA",
                "mode": "R/W",
                "leaves": {
                    "localhost:3330": {
                        "address": "localhost:3330",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    },
                    "localhost:3310": {
                        "address": "localhost:3310",
                        "status": "OFFLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    }
                }
            }
        }
    }
}
mysql-js> cluster.describe();
{
    "clusterName": "test",
    "adminType": "local",
    "defaultReplicaSet": {
        "name": "default",
        "instances": [
            {
                "name": "localhost:3310",
                "host": "localhost:3310",
                "role": "HA"
            },
            {
                "name": "localhost:3320",
                "host": "localhost:3320",
                "role": "HA"
            },
            {
                "name": "localhost:3330",
                "host": "localhost:3330",
                "role": "HA"
            }
        ]
    }
}
mysql-js> 

■ 停止したインスタンスの再開


mysql-js> dba.startLocalInstance(3310)
The MySQL sandbox instance on this host in 
/home/mysql/mysql-sandboxes/3310 will be started

Starting MySQL instance...

Instance localhost:3310 successfully started.

mysql-js> cluster.status()
{
    "clusterName": "test",
    "defaultReplicaSet": {
        "status": "Cluster is NOT tolerant to any failures.",
        "topology": {
            "localhost:3320": {
                "address": "localhost:3320",
                "status": "ONLINE",
                "role": "HA",
                "mode": "R/W",
                "leaves": {
                    "localhost:3330": {
                        "address": "localhost:3330",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    },
                    "localhost:3310": {
                        "address": "localhost:3310",
                        "status": "OFFLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    }
                }
            }
        }
    }
}
mysql-js> 

■ インスタンスをInnoDB Clusterグループへ再参加


mysql-js> dba.validateInstance('root@localhost:3310')
Please provide a password for 'root@localhost:3310': 
Validating instance...


Running check command.
Checking Group Replication prerequisites.
* Comparing options compatibility with Group Replication... PASS
Server configuration is compliant with the requirements.
* Checking server version... PASS
Server is 5.7.15

* Checking that server_id is unique... PASS
The server_id is valid.


* Checking compliance of existing tables... PASS

The instance: localhost:3310 is valid for Cluster usage

mysql-js> cluster.rejoinInstance('root@localhost:3310')
Please provide the password for 'root@localhost:3310': 
The instance will try rejoining the InnoDB cluster. Depending on the original
problem that made the instance unavailable the rejoin, operation might not be
successful and further manual steps will be needed to fix the underlying
problem.

Please monitor the output of the rejoin operation and take necessary action if
the instance cannot rejoin.
mysql-js> cluster.status()
{
    "clusterName": "test",
    "defaultReplicaSet": {
        "status": "Cluster tolerant to up to ONE failure.",
        "topology": {
            "localhost:3320": {
                "address": "localhost:3320",
                "status": "ONLINE",
                "role": "HA",
                "mode": "R/W",
                "leaves": {
                    "localhost:3310": {
                        "address": "localhost:3310",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    },
                    "localhost:3330": {
                        "address": "localhost:3330",
                        "status": "ONLINE",
                        "role": "HA",
                        "mode": "R/O",
                        "leaves": {}
                    }
                }
            }
        }
    }
}
mysql-js> 

■ SQLクライアントを利用したデータ同期と参照用インスタンスへのラウンドロビンを確認。

1) データ作成


-bash-4.2$ mysql -u root -p -h 127.0.0.1 -P 6446
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 42
Server version: 5.7.15-labs-gr090-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+-------------------------------+
| Database                      |
+-------------------------------+
| information_schema            |
| mysql                         |
| mysql_innodb_cluster_metadata |
| performance_schema            |
| sys                           |
+-------------------------------+
5 rows in set (0.00 sec)

mysql> create database test;
Query OK, 1 row affected (0.05 sec)

mysql> use test
Database changed
mysql> CREATE TABLE `InnoDB_Cluster` (
    -> `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
    -> `comment` varchar(100) NOT NULL,
    -> `last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    -> PRIMARY KEY (`id`)
    -> ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected (0.05 sec)

mysql> show databases;
+-------------------------------+
| Database                      |
+-------------------------------+
| information_schema            |
| mysql                         |
| mysql_innodb_cluster_metadata |
| performance_schema            |
| sys                           |
| test                          |
+-------------------------------+
6 rows in set (0.00 sec)

mysql> exit
Bye
-bash-4.2$ mysql -u root -p -h 127.0.0.1 -P 6447
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 21
Server version: 5.7.15-labs-gr090-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+-------------------------------+
| Database                      |
+-------------------------------+
| information_schema            |
| mysql                         |
| mysql_innodb_cluster_metadata |
| performance_schema            |
| sys                           |
| test                          |
+-------------------------------+
6 rows in set (0.00 sec)

mysql> 

■ データを1万件入れてます。

※ もし、参照側データベースにデータをINSERTしようとすると以下のようにエラーになる。
1290 (HY000): The MySQL server is running with the –super-read-only option so it cannot execute this statement
MySQL5.7からのsuper-read-onlyオプションが設定されている。

-bash-4.2$ python innodb_cluster.py 
Finish Creating Data
-bash-4.2$ 

■ 書き込み接続の確認(MySQL Routerの設定により、特定のマスターへ接続)

-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6446 -e 'select @@port,count(*) from test.InnoDB_Cluster'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+----------+
| @@port | count(*) |
+--------+----------+
|   3320 |    10000 |
+--------+----------+
-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6446 -e 'select @@port,count(*) from test.InnoDB_Cluster'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+----------+
| @@port | count(*) |
+--------+----------+
|   3320 |    10000 |
+--------+----------+
-bash-4.2$ 

■ 読み込み接続の確認(MySQL Routerの設定により、複数スレーブへのラウンドロビン接続)

-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6447 -e 'select @@port,count(*) from test.InnoDB_Cluster'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+----------+
| @@port | count(*) |
+--------+----------+
|   3330 |    10000 |
+--------+----------+
-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6447 -e 'select @@port,count(*) from test.InnoDB_Cluster'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+----------+
| @@port | count(*) |
+--------+----------+
|   3310 |    10000 |
+--------+----------+
-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6447 -e 'select @@port,count(*) from test.InnoDB_Cluster'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+----------+
| @@port | count(*) |
+--------+----------+
|   3330 |    10000 |
+--------+----------+
-bash-4.2$ 

■ その他参考:リポジトリー(各ノードで持っているので冗長化が取れている)

-bash-4.2$ mysql -u root -pP@33word -h 127.0.0.1 -P 6446
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 50
Server version: 5.7.15-labs-gr090-log MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+-------------------------------+
| Database                      |
+-------------------------------+
| information_schema            |
| mysql                         |
| mysql_innodb_cluster_metadata |
| performance_schema            |
| sys                           |
| test                          |
+-------------------------------+
6 rows in set (0.01 sec)

mysql> use mysql_innodb_cluster_metadata
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+-----------------------------------------+
| Tables_in_mysql_innodb_cluster_metadata |
+-----------------------------------------+
| clusters                                |
| hosts                                   |
| instances                               |
| replicasets                             |
| schema_version                          |
+-----------------------------------------+
5 rows in set (0.00 sec)

mysql> desc clusters;
+---------------------+------------------+------+-----+---------+----------------+
| Field               | Type             | Null | Key | Default | Extra          |
+---------------------+------------------+------+-----+---------+----------------+
| cluster_id          | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| cluster_name        | varchar(40)      | NO   | UNI | NULL    |                |
| default_replicaset  | int(10) unsigned | YES  | MUL | NULL    |                |
| description         | text             | YES  |     | NULL    |                |
| mysql_user_accounts | blob             | YES  |     | NULL    |                |
| options             | json             | YES  |     | NULL    |                |
| attributes          | json             | YES  |     | NULL    |                |
+---------------------+------------------+------+-----+---------+----------------+
7 rows in set (0.00 sec)

mysql> desc hosts;
+--------------------+------------------+------+-----+---------+----------------+
| Field              | Type             | Null | Key | Default | Extra          |
+--------------------+------------------+------+-----+---------+----------------+
| host_id            | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| host_name          | varchar(128)     | YES  |     | NULL    |                |
| ip_address         | varchar(45)      | YES  |     | NULL    |                |
| location           | varchar(256)     | NO   |     | NULL    |                |
| attributes         | json             | YES  |     | NULL    |                |
| admin_user_account | json             | YES  |     | NULL    |                |
+--------------------+------------------+------+-----+---------+----------------+
6 rows in set (0.00 sec)

mysql> desc instances;
+-------------------+---------------------------+------+-----+---------+----------------+
| Field             | Type                      | Null | Key | Default | Extra          |
+-------------------+---------------------------+------+-----+---------+----------------+
| instance_id       | int(10) unsigned          | NO   | PRI | NULL    | auto_increment |
| host_id           | int(10) unsigned          | NO   | MUL | NULL    |                |
| replicaset_id     | int(10) unsigned          | YES  | MUL | NULL    |                |
| mysql_server_uuid | varchar(40)               | NO   | UNI | NULL    |                |
| instance_name     | varchar(40)               | NO   | UNI | NULL    |                |
| role              | enum('HA','readScaleOut') | NO   |     | NULL    |                |
| weight            | float                     | YES  |     | NULL    |                |
| addresses         | json                      | NO   |     | NULL    |                |
| attributes        | json                      | YES  |     | NULL    |                |
| version_token     | int(10) unsigned          | YES  |     | NULL    |                |
| description       | text                      | YES  |     | NULL    |                |
+-------------------+---------------------------+------+-----+---------+----------------+
11 rows in set (0.00 sec)

mysql> desc replicasets;
+-----------------+------------------+------+-----+---------+----------------+
| Field           | Type             | Null | Key | Default | Extra          |
+-----------------+------------------+------+-----+---------+----------------+
| replicaset_id   | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| cluster_id      | int(10) unsigned | NO   | MUL | NULL    |                |
| replicaset_type | enum('gr')       | NO   |     | NULL    |                |
| topology_type   | enum('pm','mm')  | NO   |     | pm      |                |
| replicaset_name | varchar(40)      | NO   |     | NULL    |                |
| active          | tinyint(1)       | NO   |     | NULL    |                |
| attributes      | json             | YES  |     | NULL    |                |
| description     | text             | YES  |     | NULL    |                |
+-----------------+------------------+------+-----+---------+----------------+
8 rows in set (0.00 sec)

mysql> desc schema_version;
+-------+--------+------+-----+---------+-------+
| Field | Type   | Null | Key | Default | Extra |
+-------+--------+------+-----+---------+-------+
| major | int(1) | NO   |     | 0       |       |
| minor | int(1) | NO   |     | 0       |       |
+-------+--------+------+-----+---------+-------+
2 rows in set (0.00 sec)

mysql> 

■ その他参考:MySQL Router設定

[root@replications mysql-sandboxes]# cat /etc/mysqlrouter/mysqlrouter.conf 
[DEFAULT]
plugin_folder=/lib64/mysqlrouter
# logging_folder=/var/log/mysqlrouter

[logger]
level = INFO

[metadata_cache]
bootstrap_server_addresses=mysql://localhost:3310,mysql://localhost:3320,mysql://localhost:3330
user=mysql_innodb_cluster_reader
password=]yZQL(C7H7AU.x(2
metadata_cluster=test
ttl=300
metadata_replicaset=default

[routing:default_rw]
bind_port=6446
destinations=metadata-cache:///default?role=PRIMARY
mode=read-write

[routing:default_ro]
bind_port=6447
destinations=metadata-cache:///default?role=SECONDARY
mode=read-only
[root@replications mysql-sandboxes]# 


[root@replications mysql-sandboxes]# cat /etc/mysqlrouter/mysqlrouter.ini 
# Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA

#
# MySQL Router configuration file
#
# Documentation is available at
#    http://dev.mysql.com/doc/mysql-router/en/

[DEFAULT]
logging_folder = /var/log/mysqlrouter/
plugin_folder = /usr/lib64/mysqlrouter
runtime_folder = /var/run/mysqlrouter
config_folder = /etc/mysqlrouter

[logger]
level = info

# If no plugin is configured which starts a service, keepalive
# will make sure MySQL Router will not immediately exit. It is
# safe to remove once Router is configured.
[keepalive]
interval = 60
[root@replications mysql-sandboxes]# 

参考)
https://dev.mysql.com/doc/mysql-innodb-cluster/en/mysql-innodb-cluster-getting-started.html
http://wagnerbianchi.com/blog/?p=1229
http://wagnerbianchi.com/blog/?p=1259
https://www.pythian.com/blog/rejoining-node-mysqls-innodb-cluster/


MySQL5.7から実装された、マルチソースレプリケーションに関しての質問も少しずつ増えてきたので、改めて基本的な挙動をこちらに纏めました。
基本的には、通常のレプリケーションの挙動と変わりませんが、CHANNELに分けて1台のスレーブが複数マスターからのログを受け取るので、運用上、気を付けないといけない部分が増えてくるので、リカバリー含むPOCはしておいた方が安心です。

■ マスター側の設定(複数マスターサーバー全体でサーバーID以外は同じ)
※ マスター側は通常のレプリケーションの設定をしています。(サーバーID, バイナリーログ等)
※ ログポジションベース、GTIDモードどちらも可能ですが、今回はGTIDで設定しています。
※ Masterに予めスレーブ用のアカウントを作成済み。

Optionファイルに設定した方が良いですが検証なので、動的に設定しています。
MySQL8.0ではここら辺の動的設定も永続化するオプションが追加されています。

以下の構成での説明になります

Master1(Port:63301) --------------------->
                                            SLAVE(Port:63303)
Master2(Port:63302) --------------------->

■ Slaveにて、各マスターへの接続設定を行います。


mysql> STOP SLAVE;SET GLOBAL master_info_repository = 'TABLE';SET GLOBAL relay_log_info_repository = 'TABLE';
Query OK, 0 rows affected, 1 warning (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> CHANGE MASTER TO MASTER_HOST='localhost', MASTER_USER='multi_source', MASTER_PORT=63301, MASTER_PASSWORD='password', \
    -> MASTER_AUTO_POSITION = 1 FOR CHANNEL 'master-1';
Query OK, 0 rows affected, 1 warning (0.06 sec)

mysql> CHANGE MASTER TO MASTER_HOST='localhost', MASTER_USER='multi_source', MASTER_PORT=63302, MASTER_PASSWORD='password', \
    -> MASTER_AUTO_POSITION = 1 FOR CHANNEL 'master-2';
Query OK, 0 rows affected, 1 warning (0.04 sec)

mysql>

mysql> START SLAVE FOR CHANNEL 'master-1';START SLAVE FOR CHANNEL 'master-2';
Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> 

■ レプリケーションステータスの確認
(CHANNELを指定しなければ、全てのCHANNELの接続情報が表示されます)


mysql> SHOW SLAVE STATUS FOR CHANNEL 'master-1' \G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: localhost
                  Master_User: multi_source
                  Master_Port: 63301
                Connect_Retry: 60
              Master_Log_File: Fabric02-bin.000001
          Read_Master_Log_Pos: 885
               Relay_Log_File: Fabric02-relay-bin-master@002d1.000002
                Relay_Log_Pos: 423
        Relay_Master_Log_File: Fabric02-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 885
              Relay_Log_Space: 686
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: a0c35acf-839f-11e6-89f4-080027d65c57
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: a0c35acf-839f-11e6-89f4-080027d65c57:1-3,
a65948cd-839f-11e6-8cb5-080027d65c57:1-3,
abf190f1-839f-11e6-8dbb-080027d65c57:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-1
           Master_TLS_Version: 
1 row in set (0.00 sec)

mysql> SHOW SLAVE STATUS FOR CHANNEL 'master-2' \G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: localhost
                  Master_User: multi_source
                  Master_Port: 63302
                Connect_Retry: 60
              Master_Log_File: Fabric02-bin.000001
          Read_Master_Log_Pos: 885
               Relay_Log_File: Fabric02-relay-bin-master@002d2.000002
                Relay_Log_Pos: 1104
        Relay_Master_Log_File: Fabric02-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 885
              Relay_Log_Space: 1327
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
                  Master_UUID: a65948cd-839f-11e6-8cb5-080027d65c57
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: a65948cd-839f-11e6-8cb5-080027d65c57:1-3
            Executed_Gtid_Set: a0c35acf-839f-11e6-89f4-080027d65c57:1-3,
a65948cd-839f-11e6-8cb5-080027d65c57:1-3,
abf190f1-839f-11e6-8dbb-080027d65c57:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-2
           Master_TLS_Version: 
1 row in set (0.00 sec)

mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: localhost
                  Master_User: multi_source
                  Master_Port: 63301
                Connect_Retry: 60
              Master_Log_File: Fabric02-bin.000001
          Read_Master_Log_Pos: 2102
               Relay_Log_File: Fabric02-relay-bin-master@002d1.000002
                Relay_Log_Pos: 1640
        Relay_Master_Log_File: Fabric02-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 2102
              Relay_Log_Space: 1903
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: a0c35acf-839f-11e6-89f4-080027d65c57
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: a0c35acf-839f-11e6-89f4-080027d65c57:4-8
            Executed_Gtid_Set: a0c35acf-839f-11e6-89f4-080027d65c57:1-8,
a65948cd-839f-11e6-8cb5-080027d65c57:1-6,
abf190f1-839f-11e6-8dbb-080027d65c57:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-1
           Master_TLS_Version: 
*************************** 2. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: localhost
                  Master_User: multi_source
                  Master_Port: 63302
                Connect_Retry: 60
              Master_Log_File: Fabric02-bin.000001
          Read_Master_Log_Pos: 1623
               Relay_Log_File: Fabric02-relay-bin-master@002d2.000002
                Relay_Log_Pos: 1842
        Relay_Master_Log_File: Fabric02-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 1623
              Relay_Log_Space: 2065
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
                  Master_UUID: a65948cd-839f-11e6-8cb5-080027d65c57
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: a65948cd-839f-11e6-8cb5-080027d65c57:1-6
            Executed_Gtid_Set: a0c35acf-839f-11e6-89f4-080027d65c57:1-8,
a65948cd-839f-11e6-8cb5-080027d65c57:1-6,
abf190f1-839f-11e6-8dbb-080027d65c57:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-2
           Master_TLS_Version: 
2 rows in set (0.00 sec)

mysql> 

■ Performance_Schemaのコネクションテーブルからも接続を確認可能です。


mysql> SELECT * FROM performance_schema.replication_connection_status WHERE channel_name='master-1'\G
*************************** 1. row ***************************
             CHANNEL_NAME: master-1
               GROUP_NAME: 
              SOURCE_UUID: a0c35acf-839f-11e6-89f4-080027d65c57
                THREAD_ID: 36
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 38
 LAST_HEARTBEAT_TIMESTAMP: 2016-09-26 13:43:04
 RECEIVED_TRANSACTION_SET: a0c35acf-839f-11e6-89f4-080027d65c57:4-8
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

mysql> SELECT * FROM performance_schema.replication_connection_status WHERE channel_name='master-2'\G
*************************** 1. row ***************************
             CHANNEL_NAME: master-2
               GROUP_NAME: 
              SOURCE_UUID: a65948cd-839f-11e6-8cb5-080027d65c57
                THREAD_ID: 38
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 39
 LAST_HEARTBEAT_TIMESTAMP: 2016-09-26 13:43:01
 RECEIVED_TRANSACTION_SET: a65948cd-839f-11e6-8cb5-080027d65c57:1-6
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

mysql> 

全てのマスターからスレーブに配布するスキーマが異なっていれば、運用的な敷居は低いかと思います。
こちらの例では、マスター毎にスキーマ(データベース)が別です。

multi-master

マスターとスレーブで同じスキーマ、同じテーブルを利用する場合は、データに矛盾が発生しないようにしないといけません。
予めIDが被らないようにするなどの設計と実装が必要です。
同じスキーマを対象にしている場合のオブジェクト変更は、特定のマスターのみで変更して他のマスターではBINLOGをOFFにして変更する等の運用が必要です。

(以下の例では同じオブジェクトに別々のIDでデータをINSERTしています)
multi-source_same_schema_table

こちらは、サーバーPORTにプラスで付けてます。何でも被らなければOKです。
multi-source_same_schema_table_appli

リレーログは、各マスターサーバー毎に作成されます。
こちらの例では、relay-bin-master@002d1とrelay-bin-master@002d2でログがサーバー毎に分かれています。

-bash-4.2$ ls -l /home/mysql/fabric/mysql3/*relay*
-rw-r-----. 1 mysql mysql  223  9月 27 11:02 /home/mysql/fabric/mysql3/Fabric02-relay-bin-master@002d1.000001
-rw-r-----. 1 mysql mysql 1411  9月 27 11:16 /home/mysql/fabric/mysql3/Fabric02-relay-bin-master@002d1.000002
-rw-r-----. 1 mysql mysql   82  9月 27 11:02 /home/mysql/fabric/mysql3/Fabric02-relay-bin-master@002d1.index
-rw-r-----. 1 mysql mysql  223  9月 27 11:02 /home/mysql/fabric/mysql3/Fabric02-relay-bin-master@002d2.000001
-rw-r-----. 1 mysql mysql 1411  9月 27 11:17 /home/mysql/fabric/mysql3/Fabric02-relay-bin-master@002d2.000002
-rw-r-----. 1 mysql mysql   82  9月 27 11:02 /home/mysql/fabric/mysql3/Fabric02-relay-bin-master@002d2.index
-bash-4.2$ /usr/local/mysql57/bin/mysqlbinlog -v Fabric02-relay-bin-master@002d1.000002 | grep CREATE
GRANT SELECT, INSERT, DELETE, CREATE, DROP ON `test`.* TO 'app'@'localhost' IDENTIFIED WITH 'mysql_native_password' AS '*5BCB3E6AC345B435C7C2E6B7949A04CE6F6563D3'
CREATE TABLE MASTER01 (
-bash-4.2$ /usr/local/mysql57/bin/mysqlbinlog -v Fabric02-relay-bin-master@002d2.000002 | grep CREATE
GRANT SELECT, INSERT, DELETE, CREATE, DROP ON `test`.* TO 'app'@'localhost' IDENTIFIED WITH 'mysql_native_password' AS '*5BCB3E6AC345B435C7C2E6B7949A04CE6F6563D3'
CREATE TABLE MASTER02 (
-bash-4.2$ 

■ RESET (特定CHANNELのみRESETしてますが、CHANNELオプション無しで全てRESETも可能です。)


mysql> STOP SLAVE FOR CHANNEL 'master-1';
Query OK, 0 rows affected (0.00 sec)

mysql> RESET SLAVE FOR CHANNEL 'master-1';
Query OK, 0 rows affected (0.07 sec)

mysql> SELECT * FROM performance_schema.replication_connection_status WHERE channel_name='master-1'\G
*************************** 1. row ***************************
             CHANNEL_NAME: master-1
               GROUP_NAME: 
              SOURCE_UUID: a0c35acf-839f-11e6-89f4-080027d65c57
                THREAD_ID: NULL
            SERVICE_STATE: OFF
COUNT_RECEIVED_HEARTBEATS: 49
 LAST_HEARTBEAT_TIMESTAMP: 2016-09-26 13:48:34
 RECEIVED_TRANSACTION_SET: 
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

mysql> SELECT * FROM performance_schema.replication_connection_status WHERE channel_name='master-2'\G
*************************** 1. row ***************************
             CHANNEL_NAME: master-2
               GROUP_NAME: 
              SOURCE_UUID: a65948cd-839f-11e6-8cb5-080027d65c57
                THREAD_ID: 38
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 53
 LAST_HEARTBEAT_TIMESTAMP: 2016-09-26 13:50:01
 RECEIVED_TRANSACTION_SET: a65948cd-839f-11e6-8cb5-080027d65c57:1-6
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

mysql>

mysql> STOP SLAVE FOR CHANNEL 'master-2';
Query OK, 0 rows affected (0.00 sec)

mysql> RESET SLAVE FOR CHANNEL 'master-2';
Query OK, 0 rows affected (0.01 sec)

mysql> SELECT * FROM performance_schema.replication_connection_status WHERE channel_name='master-2'\G
*************************** 1. row ***************************
             CHANNEL_NAME: master-2
               GROUP_NAME: 
              SOURCE_UUID: a65948cd-839f-11e6-8cb5-080027d65c57
                THREAD_ID: NULL
            SERVICE_STATE: OFF
COUNT_RECEIVED_HEARTBEATS: 55
 LAST_HEARTBEAT_TIMESTAMP: 2016-09-26 13:51:01
 RECEIVED_TRANSACTION_SET: 
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.01 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: 
                  Master_Host: localhost
                  Master_User: multi_source
                  Master_Port: 63301
                Connect_Retry: 60
              Master_Log_File: 
          Read_Master_Log_Pos: 4
               Relay_Log_File: Fabric02-relay-bin-master@002d1.000002
                Relay_Log_Pos: 4
        Relay_Master_Log_File: 
             Slave_IO_Running: No
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 0
              Relay_Log_Space: 1903
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: a0c35acf-839f-11e6-89f4-080027d65c57
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: 
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: a0c35acf-839f-11e6-89f4-080027d65c57:1-8,
a65948cd-839f-11e6-8cb5-080027d65c57:1-6,
abf190f1-839f-11e6-8dbb-080027d65c57:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-1
           Master_TLS_Version: 
*************************** 2. row ***************************
               Slave_IO_State: 
                  Master_Host: localhost
                  Master_User: multi_source
                  Master_Port: 63302
                Connect_Retry: 60
              Master_Log_File: 
          Read_Master_Log_Pos: 4
               Relay_Log_File: Fabric02-relay-bin-master@002d2.000002
                Relay_Log_Pos: 4
        Relay_Master_Log_File: 
             Slave_IO_Running: No
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 0
              Relay_Log_Space: 2065
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 2
                  Master_UUID: a65948cd-839f-11e6-8cb5-080027d65c57
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: 
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: a0c35acf-839f-11e6-89f4-080027d65c57:1-8,
a65948cd-839f-11e6-8cb5-080027d65c57:1-6,
abf190f1-839f-11e6-8dbb-080027d65c57:1-2
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: master-2
           Master_TLS_Version: 
2 rows in set (0.00 sec)

mysql> 
 

参考: 
https://dev.mysql.com/doc/refman/5.7/en/replication-multi-source-overview.html
http://dev.mysql.com/doc/refman/5.7/en/replication-multi-source-tutorials.html


Oracle Public CloudにてMySQLサービスが提供開始されました。
MySQLは様々なPublic Cloudにて、既にサービスとして提供されていますが、
MySQL Enterprise版の機能を全て実装した, MySQLはこちらのサービスのみになります。

まだまだこれからの部分もありますが、今後どんどんと進化していくのでTCO削減やセキュリティ対策等で
使えそうでしたら是非活用してみて下さい。

トライアル
OPC MySQL Service詳細とトライアル
https://cloud.oracle.com/mysql
trial

プロビジョニングは非常にシンプルで、インストール時にMySQL Enterprise Monitorも同時インストール可能
provisioning

MySQL Enterprise Monitorを利用すると、MySQLの管理コスト削減をサポートしてくれます。
※Access Listで接続元を絞った運用をお勧めします。
※Access Listで他のインスタンスもモニタリング可能です。
mem

MySQL Enterprise Backupも利用可能です。
※データサイズが大きい場合に特に有効。また色々な追加機能もあります。
※データベースのバックアップ、リカバリー、ポイントインタイムリカバリーをダッシュボードから実行可能。
backup_restore

それぞれのPublic Cloudにそれそれの特徴があり、パフォーマンス、セキュリティ、可用性、安定性、管理コスト等、色々な要素を比較・検討して選択されると思います。MySQLを利用する場合は、Oracle Public CloudのMySQL Serviceも検討してみて下さい。

参考)
MySQL Enterprise Edition
https://www-jp.mysql.com/products/enterprise/


MySQL Enterprise Auditのログローテーションについての追加の手法。
合わせて、NFS上に監査ログを設定していて、万が一NFSが一時的に切れてしまった場合の挙動の確認。

1) ログサイズによる自動ローテーション
audit_log_rotate_on_size

2) mysqlutilitiesを利用したローテーション
mysqlauditadmin.exe

上記は、前回のブログ記事で紹介。
MySQL Audit Logのローテーション

今回は、CRON等で定期的に実行して、自分のカスタマイズしたいようにログをシェルでローテーションする方法です。

3) 自分でSHELLを用意して運用する方法

概要はこちらのマニュアルページで紹介しています。
https://dev.mysql.com/doc/refman/5.7/en/audit-log-logging-control.html

By default, audit_log_rotate_on_size=0 and there is no log rotation. In this case, the audit log plugin closes and reopens the log file when the audit_log_flush value changes from disabled to enabled.
If audit_log_rotate_on_size is greater than 0, setting audit_log_flush has no effect. In this case, the audit log plugin closes and reopens its log file whenever a write to the file causes its size to exceed the audit_log_rotate_on_size value.

MySQLのaudit関連設定

parameter

実際に、以下の様にSHELLを作成してログのローテーション確認してみました。
audit.logがリネームされて、新しいaudit.logファイルが再作成されています。


[root@misc01 mysql]# ls -l
合計 396
-rw-r-----. 1 mysql mysql   2813  9月 21 23:19 audit.log
-rwxr-xr-x. 1 root  root     391  9月 21 23:19 audit_log_rotate.sh
-rw-r-----. 1 mysql mysql  98304 11月 18  2015 osc_tablespace01.ibd
drwxrwxr-x. 2 mysql mysql      6  4月 29  2015 perl5
drwxrwxr-x. 2 mysql mysql   4096  4月 29  2015 ssl
-rw-r-----. 1 mysql mysql 163840  9月 19 21:43 user_tablespace01.ibd
-rw-r-----. 1 mysql mysql  65536  4月 22  2015 user_tablespace02.ibd
-rw-r-----. 1 mysql mysql  65536  4月 20  2015 user_tablespace02_8k.ibd
[root@misc01 mysql]# ./audit_log_rotate.sh 
mysql: [Warning] Using a password on the command line interface can be insecure.
[root@misc01 mysql]# ls -l
合計 400
-rw-r-----. 1 mysql mysql     47  9月 21 23:19 audit.log
-rw-r-----. 1 mysql mysql   2822  9月 21 23:19 audit_log_2016-09-21.log
-rwxr-xr-x. 1 root  root     391  9月 21 23:19 audit_log_rotate.sh
-rw-r-----. 1 mysql mysql  98304 11月 18  2015 osc_tablespace01.ibd
drwxrwxr-x. 2 mysql mysql      6  4月 29  2015 perl5
drwxrwxr-x. 2 mysql mysql   4096  4月 29  2015 ssl
-rw-r-----. 1 mysql mysql 163840  9月 19 21:43 user_tablespace01.ibd
-rw-r-----. 1 mysql mysql  65536  4月 22  2015 user_tablespace02.ibd
-rw-r-----. 1 mysql mysql  65536  4月 20  2015 user_tablespace02_8k.ibd

簡易版なので、適宜用途によって加工して利用して下さい。


[root@misc01 mysql]# cat audit_log_rotate.sh 
#!/bin/sh

#######################################
#
#    MySQL Aduit Log Rotate Shell
#
#######################################

TODAY=`date -d 'today' '+%Y-%m-%d'`
AUDIT_LOG=/home/mysql/audit_log_${TODAY}.log

# Archive and rotate audit.log
mv /home/mysql/audit.log ${AUDIT_LOG}

# Flush Audit Log for creating new log file.
mysql -u root -ppassword -e "SET GLOBAL audit_log_flush = ON;"

[root@misc01 mysql]# 


【その他、確認事項】
■ NFS上にaudit.logを配置して、運用中にNFSが切れてしまった場合の挙動の確認。

ログのローテションを確認する為に、一時的にaudit_log_rotate_on_sizeに最小限のサイズを設定しています。


root@localhost [(none)]> show variables like 'audit%';
+-----------------------------+-----------------------+
| Variable_name               | Value                 |
+-----------------------------+-----------------------+
| audit_log_buffer_size       | 1048576               |
| audit_log_connection_policy | ALL                   |
| audit_log_current_session   | OFF                   |
| audit_log_exclude_accounts  |                       |
| audit_log_file              | /home/mysql/audit.log |
| audit_log_filter_id         | 0                     |
| audit_log_flush             | OFF                   |
| audit_log_format            | NEW                   |
| audit_log_include_accounts  |                       |
| audit_log_policy            | ALL                   |
| audit_log_rotate_on_size    | 0                     |
| audit_log_statement_policy  | ALL                   |
| audit_log_strategy          | ASYNCHRONOUS          |
+-----------------------------+-----------------------+
13 rows in set (0.00 sec)

root@localhost [(none)]> set global audit_log_rotate_on_size=4096;
Query OK, 0 rows affected (0.00 sec)

root@localhost [(none)]> show variables like 'audit%';
+-----------------------------+-----------------------+
| Variable_name               | Value                 |
+-----------------------------+-----------------------+
| audit_log_buffer_size       | 1048576               |
| audit_log_connection_policy | ALL                   |
| audit_log_current_session   | OFF                   |
| audit_log_exclude_accounts  |                       |
| audit_log_file              | /home/mysql/audit.log |
| audit_log_filter_id         | 0                     |
| audit_log_flush             | OFF                   |
| audit_log_format            | NEW                   |
| audit_log_include_accounts  |                       |
| audit_log_policy            | ALL                   |
| audit_log_rotate_on_size    | 4096                  |
| audit_log_statement_policy  | ALL                   |
| audit_log_strategy          | ASYNCHRONOUS          |
+-----------------------------+-----------------------+
13 rows in set (0.01 sec)

root@localhost [(none)]> 


ログをリネームして一時的にMySQLからファイルを見えなくしています。この間にAudit対象のQueryを実行して、ローテーションサイズを超えてもadit.logが無いので監査ログは書き込まれません。


[root@misc01 mysql]# ls -l
合計 424
-rw-r-----. 1 mysql mysql    504  9月 22 21:52 audit.log
-rw-r-----. 1 mysql mysql   4466  9月 22 21:52 audit.log.14745487509667319.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487651271064.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487704909257.xml
-rw-r-----. 1 mysql mysql   2822  9月 21 23:19 audit_log_2016-09-21.log
-rwxr-xr-x. 1 root  root     391  9月 21 23:19 audit_log_rotate.sh
-rw-r-----. 1 mysql mysql  98304 11月 18  2015 osc_tablespace01.ibd
drwxrwxr-x. 2 mysql mysql      6  4月 29  2015 perl5
drwxrwxr-x. 2 mysql mysql   4096  4月 29  2015 ssl
-rw-r-----. 1 mysql mysql 163840  9月 19 21:43 user_tablespace01.ibd
-rw-r-----. 1 mysql mysql  65536  4月 22  2015 user_tablespace02.ibd
-rw-r-----. 1 mysql mysql  65536  4月 20  2015 user_tablespace02_8k.ibd

[root@misc01 mysql]# mv audit.log on_purpose_rename_audit.log
[root@misc01 mysql]# ls -l
合計 424
-rw-r-----. 1 mysql mysql   4466  9月 22 21:52 audit.log.14745487509667319.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487651271064.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487704909257.xml
-rw-r-----. 1 mysql mysql   2822  9月 21 23:19 audit_log_2016-09-21.log
-rwxr-xr-x. 1 root  root     391  9月 21 23:19 audit_log_rotate.sh
-rw-r-----. 1 mysql mysql    504  9月 22 21:52 on_purpose_rename_audit.log
-rw-r-----. 1 mysql mysql  98304 11月 18  2015 osc_tablespace01.ibd
drwxrwxr-x. 2 mysql mysql      6  4月 29  2015 perl5
drwxrwxr-x. 2 mysql mysql   4096  4月 29  2015 ssl
-rw-r-----. 1 mysql mysql 163840  9月 19 21:43 user_tablespace01.ibd
-rw-r-----. 1 mysql mysql  65536  4月 22  2015 user_tablespace02.ibd
-rw-r-----. 1 mysql mysql  65536  4月 20  2015 user_tablespace02_8k.ibd
[root@misc01 mysql]# ls -l
合計 424
-rw-r-----. 1 mysql mysql   4466  9月 22 21:52 audit.log.14745487509667319.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487651271064.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487704909257.xml
-rw-r-----. 1 mysql mysql   2822  9月 21 23:19 audit_log_2016-09-21.log
-rwxr-xr-x. 1 root  root     391  9月 21 23:19 audit_log_rotate.sh
-rw-r-----. 1 mysql mysql    961  9月 22 21:53 on_purpose_rename_audit.log
-rw-r-----. 1 mysql mysql  98304 11月 18  2015 osc_tablespace01.ibd
drwxrwxr-x. 2 mysql mysql      6  4月 29  2015 perl5
drwxrwxr-x. 2 mysql mysql   4096  4月 29  2015 ssl
-rw-r-----. 1 mysql mysql 163840  9月 19 21:43 user_tablespace01.ibd
-rw-r-----. 1 mysql mysql  65536  4月 22  2015 user_tablespace02.ibd
-rw-r-----. 1 mysql mysql  65536  4月 20  2015 user_tablespace02_8k.ibd
[root@misc01 mysql]# ls -l
合計 428
-rw-r-----. 1 mysql mysql   4466  9月 22 21:52 audit.log.14745487509667319.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487651271064.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487704909257.xml
-rw-r-----. 1 mysql mysql   2822  9月 21 23:19 audit_log_2016-09-21.log
-rwxr-xr-x. 1 root  root     391  9月 21 23:19 audit_log_rotate.sh
-rw-r-----. 1 mysql mysql   6902  9月 22 21:53 on_purpose_rename_audit.log
-rw-r-----. 1 mysql mysql  98304 11月 18  2015 osc_tablespace01.ibd
drwxrwxr-x. 2 mysql mysql      6  4月 29  2015 perl5
drwxrwxr-x. 2 mysql mysql   4096  4月 29  2015 ssl
-rw-r-----. 1 mysql mysql 163840  9月 19 21:43 user_tablespace01.ibd
-rw-r-----. 1 mysql mysql  65536  4月 22  2015 user_tablespace02.ibd
-rw-r-----. 1 mysql mysql  65536  4月 20  2015 user_tablespace02_8k.ibd

NFSの接続が戻ったと仮定して、ログを元の名前に戻す。
21:56の段階でaudit.logが無かった時のデータがaudit.log.14745489750144595.xmlに書き込まれている。
ログローテーションのサイズ(4096)をオーバーしているがデータがロストしていない事が確認出来る。
また、新規でaudit.logが作成されている。ローテーションも設定サイズできちんと行われている。
audit.logに書き込めない間は、audit_log_bufferに蓄積されているようです。


[root@misc01 mysql]# mv on_purpose_rename_audit.log audit.log
[root@misc01 mysql]# ls -l
合計 440
-rw-r-----. 1 mysql mysql   1418  9月 22 21:56 audit.log
-rw-r-----. 1 mysql mysql   4466  9月 22 21:52 audit.log.14745487509667319.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487651271064.xml
-rw-r-----. 1 mysql mysql   4169  9月 22 21:52 audit.log.14745487704909257.xml
-rw-r-----. 1 mysql mysql  15594  9月 22 21:56 audit.log.14745489750144595.xml
-rw-r-----. 1 mysql mysql   2822  9月 21 23:19 audit_log_2016-09-21.log
-rwxr-xr-x. 1 root  root     391  9月 21 23:19 audit_log_rotate.sh
-rw-r-----. 1 mysql mysql  98304 11月 18  2015 osc_tablespace01.ibd
drwxrwxr-x. 2 mysql mysql      6  4月 29  2015 perl5
drwxrwxr-x. 2 mysql mysql   4096  4月 29  2015 ssl
-rw-r-----. 1 mysql mysql 163840  9月 19 21:43 user_tablespace01.ibd
-rw-r-----. 1 mysql mysql  65536  4月 22  2015 user_tablespace02.ibd
-rw-r-----. 1 mysql mysql  65536  4月 20  2015 user_tablespace02_8k.ibd
[root@misc01 mysql]# 

nfs


補足:ログに書けない間も、MySQLのステータス変数はカウントアップされている事がaudit_log_current_sizeから確認する事が出来ます。
ファイルを戻したタイミングで、ログがファイルに書き込まれ、audit_log_current_sizeは値が小さくなっている事が確認出来ます。

root@localhost [(none)]> show status like 'audit%';
+-------------------------------+-------+
| Variable_name                 | Value |
+-------------------------------+-------+
| audit_log_current_size        | 15585 |
| audit_log_event_max_drop_size | 0     |
| audit_log_events              | 1     |
| audit_log_events_buffered     | 0     |
| audit_log_events_filtered     | 0     |
| audit_log_events_lost         | 0     |
| audit_log_events_written      | 59    |
| audit_log_total_size          | 28389 |
| audit_log_write_waits         | 0     |
+-------------------------------+-------+
9 rows in set (0.00 sec)

root@localhost [(none)]> show status like 'audit%';
+-------------------------------+-------+
| Variable_name                 | Value |
+-------------------------------+-------+
| audit_log_current_size        | 1418  |
| audit_log_event_max_drop_size | 0     |
| audit_log_events              | 1     |
| audit_log_events_buffered     | 0     |
| audit_log_events_filtered     | 0     |
| audit_log_events_lost         | 0     |
| audit_log_events_written      | 62    |
| audit_log_total_size          | 29816 |
| audit_log_write_waits         | 0     |
+-------------------------------+-------+
9 rows in set (0.00 sec)

root@localhost [(none)]> 


MySQL8.0がDockerリポジトリーで提供されているので、Dockerに8.0のイメージをダウンロードしてインストールしてみました。
同時にMySQL8.0の新機能確認として、動的にGlobal Variablesを変更して永続化出来るか?また、データディクショナリの状況も確認してみました。

■ DockerレポジトリーとイメージTag
https://hub.docker.com/r/mysql/mysql-server/
https://hub.docker.com/r/mysql/mysql-server/tags/

MySQL8.0 Dockerイメージダウンロード~起動まで


[root@DockerHost oracle]# docker pull mysql/mysql-server:8.0
8.0: Pulling from mysql/mysql-server
7f369f1cac0b: Pull complete 
897fddccf3d8: Pull complete 
865c22dab1e4: Pull complete 
3e61c960af44: Pull complete 
fcd95ea99f45: Pull complete 
e9cae96efb21: Pull complete 
671450fab9a5: Pull complete 
1b8291aef5a7: Pull complete 
99ef814fb233: Pull complete 
c1add8e582f0: Pull complete 
Digest: sha256:c65b1da17c01749a28fc0b0865f94bf02053f290f23a28bf4fe9d8447dccadd6
Status: Downloaded newer image for mysql/mysql-server:8.0
[root@DockerHost oracle]# docker images
REPOSITORY           TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
mysql/mysql-server   8.0                 c1add8e582f0        5 days ago          393.4 MB
mysql/mysql-server   5.7.12              4e1d42e32c43        5 months ago        296.7 MB
mysql                5.7.10              ea0aca21950d        9 months ago        360.3 MB
mysql/mysql-server   5.7.10              e472f1765697        9 months ago        294.6 MB
[root@DockerHost oracle]# 

[root@DockerHost oracle]# docker run --name mysql8 -v /docker/docker8:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=mysql -d mysql/mysql-server:8.0
f0d7a1a715633b76aaadbba8b8fa6a63b293a9a88d193d28cd44cebd33d08955
[root@DockerHost oracle]# 

[root@DockerHost oracle]# docker ps -a
CONTAINER ID        IMAGE                       COMMAND                  CREATED              STATUS                    PORTS                 NAMES
f0d7a1a71563        mysql/mysql-server:8.0      "/entrypoint.sh mysql"   About a minute ago   Up About a minute         3306/tcp, 33060/tcp   mysql8
88da7fe02e00        mysql:5.7.10                "/entrypoint.sh mysql"   3 months ago         Exited (0) 3 months ago                         my_docker03
4fde03dc4cb5        mysql/mysql-server:5.7.10   "/entrypoint.sh mysql"   8 months ago         Exited (0) 3 months ago                         my_docker02
5152a22b2aa0        mysql:5.7.10                "/entrypoint.sh mysql"   8 months ago         Exited (0) 8 months ago                         my_docker01
[root@DockerHost oracle]# 

[root@DockerHost oracle]# docker inspect -f "{{.Config.Hostname}}, {{.NetworkSettings.IPAddress}}" $(docker ps | grep -v "^CONTAINER" | awk '{print $1}')
f0d7a1a71563, 172.17.0.2
[root@DockerHost oracle]# 

ログインしてバージョン確認

[root@DockerHost oracle]# docker exec -it mysql8 mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 8.0.0-dmr MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> select @@version;
+-----------+
| @@version |
+-----------+
| 8.0.0-dmr |
+-----------+
1 row in set (0.00 sec)

mysql> 

■ SET PERSIST Statementの確認
http://mysqlserverteam.com/mysql-8-0-persisting-configuration-variables/


mysql> SELECT * FROM performance_schema.variables_info WHERE variable_source != 'COMPILED';
+--------------------+-----------------+---------------+-----------+-----------+
| VARIABLE_NAME      | VARIABLE_SOURCE | VARIABLE_PATH | MIN_VALUE | MAX_VALUE |
+--------------------+-----------------+---------------+-----------+-----------+
| datadir            | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| foreign_key_checks | DYNAMIC         |               | 0         | 0         |
| log_error          | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| pid_file           | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| secure_file_priv   | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| skip_name_resolve  | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| socket             | GLOBAL          | /etc/my.cnf   | 0         | 0         |
+--------------------+-----------------+---------------+-----------+-----------+
7 rows in set (0.00 sec)

mysql> show variables like 'log_timestamps';                                               
+----------------+-------+
| Variable_name  | Value |
+----------------+-------+
| log_timestamps | UTC   |
+----------------+-------+
1 row in set (0.01 sec)

mysql> SET PERSIST log_timestamps='SYSTEM';
Query OK, 0 rows affected (0.00 sec)

mysql> show variables like 'log_timestamps';                                               
+----------------+--------+
| Variable_name  | Value  |
+----------------+--------+
| log_timestamps | SYSTEM |
+----------------+--------+
1 row in set (0.00 sec)

mysql> SELECT * FROM performance_schema.variables_info WHERE variable_source != 'COMPILED';
+--------------------+-----------------+---------------+-----------+-----------+
| VARIABLE_NAME      | VARIABLE_SOURCE | VARIABLE_PATH | MIN_VALUE | MAX_VALUE |
+--------------------+-----------------+---------------+-----------+-----------+
| datadir            | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| foreign_key_checks | DYNAMIC         |               | 0         | 0         |
| log_error          | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| log_timestamps     | DYNAMIC         |               | 0         | 0         |
| pid_file           | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| secure_file_priv   | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| skip_name_resolve  | GLOBAL          | /etc/my.cnf   | 0         | 0         |
| socket             | GLOBAL          | /etc/my.cnf   | 0         | 0         |
+--------------------+-----------------+---------------+-----------+-----------+
8 rows in set (0.00 sec)

mysql> 

再起動後も値が反映されている事を確認


[root@DockerHost oracle]# docker stop f0d7a1a71563
f0d7a1a71563
[root@DockerHost oracle]# docker ps -a
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                     PORTS               NAMES
f0d7a1a71563        mysql/mysql-server:8.0      "/entrypoint.sh mysql"   11 minutes ago      Exited (0) 2 seconds ago                       mysql8
88da7fe02e00        mysql:5.7.10                "/entrypoint.sh mysql"   3 months ago        Exited (0) 3 months ago                        my_docker03
4fde03dc4cb5        mysql/mysql-server:5.7.10   "/entrypoint.sh mysql"   8 months ago        Exited (0) 3 months ago                        my_docker02
5152a22b2aa0        mysql:5.7.10                "/entrypoint.sh mysql"   8 months ago        Exited (0) 8 months ago                        my_docker01
[root@DockerHost oracle]# docker start f0d7a1a71563
f0d7a1a71563
[root@DockerHost oracle]# docker ps -a
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS                    PORTS                 NAMES
f0d7a1a71563        mysql/mysql-server:8.0      "/entrypoint.sh mysql"   11 minutes ago      Up 2 seconds              3306/tcp, 33060/tcp   mysql8
88da7fe02e00        mysql:5.7.10                "/entrypoint.sh mysql"   3 months ago        Exited (0) 3 months ago                         my_docker03
4fde03dc4cb5        mysql/mysql-server:5.7.10   "/entrypoint.sh mysql"   8 months ago        Exited (0) 3 months ago                         my_docker02
5152a22b2aa0        mysql:5.7.10                "/entrypoint.sh mysql"   8 months ago        Exited (0) 8 months ago                         my_docker01
[root@DockerHost oracle]# 

[root@DockerHost oracle]# docker exec -it mysql8 mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 8.0.0-dmr MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show variables like 'log_timestamps';
+----------------+--------+
| Variable_name  | Value  |
+----------------+--------+
| log_timestamps | SYSTEM |
+----------------+--------+
1 row in set (0.02 sec)

mysql> 

(補足) SET PERSISTで設定変更した、GLOBAL変数は以下のようにmysqld-auto.cnfから読み込まれています。

Under the hood the settings will be persisted to a file named mysqld-auto.cnf which will be created in the data directory.
This file will be read during server startup just like any other configuration file, and all variables present in this file will be applied as the highest priority.
That means the file mysqld-auto.cnf will be the last file to be applied on server startup (even after command-line options) and takes precedence if a specific setting has been specified in more than one location.


mysql>  select VARIABLE_NAME,VARIABLE_SOURCE,VARIABLE_PATH from performance_schema.variables_info where VARIABLE_PATH <> '';
+-------------------+-----------------+--------------------------------+
| VARIABLE_NAME     | VARIABLE_SOURCE | VARIABLE_PATH                  |
+-------------------+-----------------+--------------------------------+
| datadir           | GLOBAL          | /etc/my.cnf                    |
| log_error         | GLOBAL          | /etc/my.cnf                    |
| log_timestamps    | PERSISTED       | /var/lib/mysql/mysqld-auto.cnf |
| pid_file          | GLOBAL          | /etc/my.cnf                    |
| secure_file_priv  | GLOBAL          | /etc/my.cnf                    |
| skip_name_resolve | GLOBAL          | /etc/my.cnf                    |
| socket            | GLOBAL          | /etc/my.cnf                    |
+-------------------+-----------------+--------------------------------+
7 rows in set (0.00 sec)

mysql> 

■MySQL8.0データディクショナリの確認


[root@DockerHost var]# docker exec -it mysql8 mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 8.0.0-dmr MySQL Community Server (GPL)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> source /var/lib/mysql/init-docker-sakila.sql

Query OK, 200 rows affected (0.01 sec)
Records: 200  Duplicates: 0  Warnings: 0

Query OK, 603 rows affected (0.04 sec)
Records: 603  Duplicates: 0  Warnings: 0

Query OK, 600 rows affected (0.01 sec)
Records: 600  Duplicates: 0  Warnings: 0

<SNIP>

Query OK, 109 rows affected (0.01 sec)
Records: 109  Duplicates: 0  Warnings: 0

Query OK, 599 rows affected (0.04 sec)
Records: 599  Duplicates: 0  Warnings: 0

Query OK, 1000 rows affected (0.17 sec)
Records: 1000  Duplicates: 0  Warnings: 0

Query OK, 5462 rows affected (0.18 sec)
Records: 5462  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sakila             |
| sys                |
+--------------------+
5 rows in set (0.01 sec)

mysql> 

■ MySQL8.0で作成したSakilaサンプルデータベースのデータディクショナリー確認

[root@DockerHost sakila]# ls -l /docker/docker8/sakila
total 25104
-rw-r----- 1 27 27   147456 Sep 20 13:55 actor.ibd
-rw-r----- 1 27 27   278528 Sep 20 13:55 address.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 category.ibd
-rw-r----- 1 27 27   147456 Sep 20 13:55 city.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 country.ibd
-rw-r----- 1 27 27   229376 Sep 20 13:55 customer.ibd
-rw-r----- 1 27 27   376832 Sep 20 13:55 film_actor.ibd
-rw-r----- 1 27 27   180224 Sep 20 13:55 film_category.ibd
-rw-r----- 1 27 27   376832 Sep 20 13:55 film.ibd
-rw-r----- 1 27 27   294912 Sep 20 13:55 film_text.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_00000000000000ae_INDEX_1.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_00000000000000ae_INDEX_2.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_00000000000000ae_INDEX_3.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_00000000000000ae_INDEX_4.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_00000000000000ae_INDEX_5.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_00000000000000ae_INDEX_6.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_BEING_DELETED_CACHE.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_BEING_DELETED.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_CONFIG.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_DELETED_CACHE.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 FTS_000000000000005b_DELETED.ibd
-rw-r----- 1 27 27   475136 Sep 20 13:56 inventory.ibd
-rw-r----- 1 27 27   131072 Sep 20 13:55 language.ibd
-rw-r----- 1 27 27 10485760 Sep 20 13:56 payment.ibd
-rw-r----- 1 27 27 10485760 Sep 20 13:56 rental.ibd
-rw-r----- 1 27 27   180224 Sep 20 13:55 staff.ibd
-rw-r----- 1 27 27   163840 Sep 20 13:55 store.ibd
[root@DockerHost sakila]# 

■ MySQL5.7で作成したSakilaサンプルデータベースのデータディクショナリー確認
frmやTRGファイルが存在しています。


[root@DockerHost sakila]# ls -l /docker/docker03/sakila/
total 24620
-rw-r----- 1 999 999     8694 Jun  7 07:17 actor.frm
-rw-r----- 1 999 999   114688 Jun  7 07:17 actor.ibd
-rw-r----- 1 999 999     2863 Jun  7 07:17 actor_info.frm
-rw-r----- 1 999 999     8878 Jun  7 07:17 address.frm
-rw-r----- 1 999 999   245760 Jun  7 07:17 address.ibd
-rw-r----- 1 999 999     8648 Jun  7 07:17 category.frm
-rw-r----- 1 999 999    98304 Jun  7 07:17 category.ibd
-rw-r----- 1 999 999     8682 Jun  7 07:17 city.frm
-rw-r----- 1 999 999   114688 Jun  7 07:17 city.ibd
-rw-r----- 1 999 999     8652 Jun  7 07:17 country.frm
-rw-r----- 1 999 999    98304 Jun  7 07:17 country.ibd
-rw-r----- 1 999 999       40 Jun  7 07:17 customer_create_date.TRN
-rw-r----- 1 999 999     8890 Jun  7 07:17 customer.frm
-rw-r----- 1 999 999   196608 Jun  7 07:17 customer.ibd
-rw-r----- 1 999 999     1892 Jun  7 07:17 customer_list.frm
-rw-r----- 1 999 999      300 Jun  7 07:17 customer.TRG
-rw-r----- 1 999 999       61 Jun  7 07:17 db.opt
-rw-r----- 1 999 999       36 Jun  7 07:17 del_film.TRN
-rw-r----- 1 999 999     8648 Jun  7 07:17 film_actor.frm
-rw-r----- 1 999 999   344064 Jun  7 07:17 film_actor.ibd
-rw-r----- 1 999 999     8654 Jun  7 07:17 film_category.frm
-rw-r----- 1 999 999   147456 Jun  7 07:17 film_category.ibd
-rw-r----- 1 999 999     9188 Jun  7 07:17 film.frm
-rw-r----- 1 999 999   344064 Jun  7 07:17 film.ibd
-rw-r----- 1 999 999     2616 Jun  7 07:17 film_list.frm
-rw-r----- 1 999 999     8642 Jun  7 07:17 film_text.frm
-rw-r----- 1 999 999   262144 Jun  7 07:17 film_text.ibd
-rw-r----- 1 999 999     1093 Jun  7 07:17 film.TRG
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_0000000000000045_INDEX_1.ibd
-rw-r----- 1 999 999   131072 Jun  7 07:17 FTS_0000000000000035_0000000000000045_INDEX_2.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_0000000000000045_INDEX_3.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_0000000000000045_INDEX_4.ibd
-rw-r----- 1 999 999   131072 Jun  7 07:17 FTS_0000000000000035_0000000000000045_INDEX_5.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_0000000000000045_INDEX_6.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_BEING_DELETED_CACHE.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_BEING_DELETED.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_CONFIG.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_DELETED_CACHE.ibd
-rw-r----- 1 999 999    98304 Jun  7 07:17 FTS_0000000000000035_DELETED.ibd
-rw-r----- 1 999 999       36 Jun  7 07:17 ins_film.TRN
-rw-r----- 1 999 999     8694 Jun  7 07:17 inventory.frm
-rw-r----- 1 999 999   442368 Jun  7 07:17 inventory.ibd
-rw-r----- 1 999 999     8648 Jun  7 07:17 language.frm
-rw-r----- 1 999 999    98304 Jun  7 07:17 language.ibd
-rw-r----- 1 999 999     3234 Jun  7 07:17 nicer_but_slower_film_list.frm
-rw-r----- 1 999 999       39 Jun  7 07:17 payment_date.TRN
-rw-r----- 1 999 999     8818 Jun  7 07:17 payment.frm
-rw-r----- 1 999 999 10485760 Jun  7 07:17 payment.ibd
-rw-r----- 1 999 999      292 Jun  7 07:17 payment.TRG
-rw-r----- 1 999 999       38 Jun  7 07:17 rental_date.TRN
-rw-r----- 1 999 999     8830 Jun  7 07:17 rental.frm
-rw-r----- 1 999 999 10485760 Jun  7 07:17 rental.ibd
-rw-r----- 1 999 999      289 Jun  7 07:17 rental.TRG
-rw-r----- 1 999 999     1669 Jun  7 07:17 sales_by_film_category.frm
-rw-r----- 1 999 999     2344 Jun  7 07:17 sales_by_store.frm
-rw-r----- 1 999 999     8952 Jun  7 07:17 staff.frm
-rw-r----- 1 999 999   147456 Jun  7 07:17 staff.ibd
-rw-r----- 1 999 999     1705 Jun  7 07:17 staff_list.frm
-rw-r----- 1 999 999     8708 Jun  7 07:17 store.frm
-rw-r----- 1 999 999   131072 Jun  7 07:17 store.ibd
-rw-r----- 1 999 999       36 Jun  7 07:17 upd_film.TRN

■ 参照
15.6 Data Dictionary Usage Differences
The MySQL 8.0.0 Milestone Release is available


MySQL Clusterの検証をIBM(Softlayer)さん、株式会社インフィニットループさんと共同で行って、
結果を先日のCEDECで発表した時の資料です。インフィニットループさんがOpenSourceのゲームで検証してくれているので、内容的には参考になるかと思います。

1. MySQL Clusterを稼働させるにはネットワークが非常に重要
2. MySQL Clusterを稼働させる場合、仮想環境のCommit Ratioがコントロール出来ない場合はベアメタルが良い。
3. 仮想、ベアメタル共にSSDを利用していてもISCSIよりもやはりローカルの方が断然良い。
4. MaxNoOfExecutionThreadsより、ThreadConfigで丁寧に調整した方がパフォーマンス良い(この検証では1.2倍)
5. 今回のSYSBENCH0.5での検証では、データの偏りがあったので思うようにパフォーマンスが出なかった。(パーティションで対応可能)
6・ Sysbench0.5のOLTP R/Wの処理は、5割程がMySQL Cluster向きで残りの5割はInnDB向きの処理。(データノードを跨る処理が多かった)

今回は2週間でしたが、もう少し時間があればより最適なパラメータ構成を設定出来たかと思います。
皆さんも利用する場合は、事前に最適なパラメータ確認を実施して下さい。

MySQL Cluster Community Edition Download
http://dev.mysql.com/downloads/cluster/

Commercial Edition追加ツールとサポート
https://www-jp.mysql.com/products/cluster/


MySQL5.7.14のGroup Replication DMR版が先月リリースされていたので、基本的な動作確認をしました。
まだ、DMR版なので前回の5.7.10とまた微妙に異なっていて、
group_replication_peer_addressesがgroup_replication_group_seedsに代わっていたり、
group_replication_recovery_userやgroup_replication_recovery_passwordが無くなっていたりします。

MySQL Group Replication: A Quick Start Guide
http://mysqlhighavailability.com/mysql-group-replication-a-quick-start-guide/

MySQL5.7.10版での検証: MYSQL GROUP REPLICATION
http://variable.jp/2016/07/18/mysql-group-replication/

検証環境のオプションファイル設定
基本的にはserver_idをノード間で変更しているのみ。

# Binary logging and Replication
server_id                      = 1
log_bin                        = mysql-bin
binlog_cache_size              = 1M
binlog_stmt_cache_size         = 1M                                  # Since 5.5
max_binlog_size                = 64M                                 # Make bigger for high traffic to reduce number of files
sync_binlog                    = 0                                   # Set to 1 or higher to reduce potential loss of binary-log data
expire_logs_days               = 30                                  # We will survive easter holidays
#binlog_format                 = MIXED                               # Use MIXED if you experience some troubles
binlog_format                  = ROW
binlog_row_image               = MINIMAL                             # Since 5.6
binlog_rows_query_log_events   = 1                                   # Log Statement in ROW Base Binlog
# auto_increment_increment       = 1                                 # For Master/Master set-ups use 1 and 2
# auto_increment_offset          = 2                                 # For Master/Master set-ups use 2 for both nodes
log_timestamps = SYSTEM


# Add for GTID Mode
gtid-mode                      = on
enforce-gtid-consistency       = on
log-slave-updates

master_info_repository        = TABLE
relay_log_info_repository     = TABLE

# Group replication
# plugin-load = group_replication.so
group_replication_start_on_boot = ON
group_replication_bootstrap_group = OFF
# transaction-write-set-extraction=MURMUR32
transaction-write-set-extraction=XXHASH64
binlog-checksum=NONE
group_replication = FORCE_PLUS_PERMANENT
transaction-write-set-extraction = XXHASH64
group_replication_group_name= "00000000-1111-2222-3333-123456789ABC"
#group_replication_recovery_user='rpl_user'                             #commented out at upgrading to 5.7.14
#group_replication_recovery_password='rpl_pass'                         #commented out at upgrading to 5.7.14
group_replication_recovery_retry_count= 2
group_replication_recovery_reconnect_interval= 120
group_replication_local_address="192.168.56.101:6606"
#group_replication_peer_addresses= "192.168.56.101:18620,192.168.56.102:18620" #commented out at upgrading to 5.7.14
group_replication_group_seeds =  "192.168.56.101:6606,192.168.56.102:6606"

# 5.7.14
relay-log-recovery = ON
slave-parallel-type = LOGICAL_CLOCK
slave-preserve-commit-order = ON
slave-rows-search-algorithms = 'INDEX_SCAN,HASH_SCAN'
slave-type-conversions = ALL_NON_LOSSY
sync-master-info = 1000
sync-relay-log = 1000
slave-parallel-workers = 0

■Boot Strap Nodeの起動

– It will not try and participate in any group communication when starting but will instead configure the group as consisting only of itself.
– Any subsequent member that attempts to join the group will sync itself up with the state of this instance.

We need to pick one member and declare that it is the bootstrap node by setting group_replication_bootstrap_group=ON
Just remember to turn group_replication_bootstrap_group=OFF again after the first member is up.

root@localhost [GR_TEST]> select @@version,@@hostname;
+-----------------------+------------+
| @@version             | @@hostname |
+-----------------------+------------+
| 5.7.14-labs-gr080-log | misc01     |
+-----------------------+------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | OFFLINE      |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> show variables like 'group_replication_bootstrap_group';
+-----------------------------------+-------+
| Variable_name                     | Value |
+-----------------------------------+-------+
| group_replication_bootstrap_group | OFF   |
+-----------------------------------+-------+
1 row in set (0.03 sec)

root@localhost [GR_TEST]> SET GLOBAL group_replication_bootstrap_group= 1;
Query OK, 0 rows affected (0.00 sec)

root@localhost [GR_TEST]> show variables like 'group_replication_bootstrap_group';
+-----------------------------------+-------+
| Variable_name                     | Value |
+-----------------------------------+-------+
| group_replication_bootstrap_group | ON    |
+-----------------------------------+-------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> START GROUP_REPLICATION;
Query OK, 0 rows affected (1.07 sec)

root@localhost [GR_TEST]> SET GLOBAL group_replication_bootstrap_group= 0;
Query OK, 0 rows affected (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> select * from performance_schema.replication_group_member_stats\G
*************************** 1. row ***************************
                      CHANNEL_NAME: group_replication_applier
                           VIEW_ID: 14732520384730018:1
                         MEMBER_ID: 29ea17bc-3848-11e6-9900-0800279ca844
       COUNT_TRANSACTIONS_IN_QUEUE: 0
        COUNT_TRANSACTIONS_CHECKED: 0
          COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS: 
    LAST_CONFLICT_FREE_TRANSACTION: 
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 

■2番目のノード追加
Specify valid MySQL credentials that this node will use when requesting GTIDs from existing members of the
group (a seed or donor) necessary to perform an automated recovery (such as when joining the group, which we’ll do next):
CHANGE MASTER TO MASTER_USER=’rpl_user’, MASTER_PASSWORD=’rpl_pass’ FOR CHANNEL ‘group_replication_recovery’;
Have the new node join to become a member with: STOP GROUP_REPLICATION; START GROUP_REPLICATION; (STOP is necessary because we have start_on_boot enabled).

root@localhost [GR_TEST]>  select @@version,@@hostname;
+-----------------------+------------+
| @@version             | @@hostname |
+-----------------------+------------+
| 5.7.14-labs-gr080-log | misc02     |
+-----------------------+------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 5b07d5d8-4057-11e6-a315-0800279cea3c | misc02      |        3306 | OFFLINE      |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> CHANGE MASTER TO MASTER_USER='rpl_user', MASTER_PASSWORD='rpl_pass' FOR CHANNEL 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.02 sec)

root@localhost [GR_TEST]> STOP GROUP_REPLICATION;START GROUP_REPLICATION;
Query OK, 0 rows affected (0.00 sec)

Query OK, 0 rows affected (7.04 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | ONLINE       |
| group_replication_applier | 5b07d5d8-4057-11e6-a315-0800279cea3c | misc02      |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> select * from performance_schema.replication_group_member_stats\G
*************************** 1. row ***************************
                      CHANNEL_NAME: group_replication_applier
                           VIEW_ID: 14732520384730018:2
                         MEMBER_ID: 5b07d5d8-4057-11e6-a315-0800279cea3c
       COUNT_TRANSACTIONS_IN_QUEUE: 0
        COUNT_TRANSACTIONS_CHECKED: 0
          COUNT_CONFLICTS_DETECTED: 0
COUNT_TRANSACTIONS_ROWS_VALIDATING: 0
TRANSACTIONS_COMMITTED_ALL_MEMBERS: 
    LAST_CONFLICT_FREE_TRANSACTION: 
1 row in set (0.00 sec)

root@localhost [GR_TEST]> select user_name, user_password from mysql.slave_master_info where channel_name = 'group_replication_recovery';
+-----------+---------------+
| user_name | user_password |
+-----------+---------------+
| rpl_user  | rpl_pass      |
+-----------+---------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 

■ 追加の確認と設定(NODE2を追加後にBootstrapノードで実行)

You should also now execute step 3 (the CHANGE MASTER TO) on the node we used to bootstrap the group, if you haven’t previously done so.

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | ONLINE       |
| group_replication_applier | 5b07d5d8-4057-11e6-a315-0800279cea3c | misc02      |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> CHANGE MASTER TO MASTER_USER='rpl_user', MASTER_PASSWORD='rpl_pass' FOR CHANNEL 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.02 sec)

root@localhost [GR_TEST]> select user_name, user_password from mysql.slave_master_info where channel_name = 'group_replication_recovery';
+-----------+---------------+
| user_name | user_password |
+-----------+---------------+
| rpl_user  | rpl_pass      |
+-----------+---------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 

以下、基本動作確認

■マルチマスターの基本動作確認 (NODE1とNODE2からINSERT)

NODE1)

root@localhost [GR_TEST]> desc T01;
+--------------+-------------+------+-----+-------------------+-------+
| Field        | Type        | Null | Key | Default           | Extra |
+--------------+-------------+------+-----+-------------------+-------+
| ID           | int(11)     | NO   | PRI | NULL              |       |
| MEMO         | varchar(30) | NO   |     |                   |       |
| created_time | datetime    | YES  |     | CURRENT_TIMESTAMP |       |
+--------------+-------------+------+-----+-------------------+-------+
3 rows in set (0.01 sec)

root@localhost [GR_TEST]> insert into T01(ID,MEMO) values(1,@@hostname);
Query OK, 1 row affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
+----+--------+---------------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE2)

root@localhost [GR_TEST]> select * from T01;
Empty set (0.00 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
+----+--------+---------------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> insert into T01(ID,MEMO) values(2,@@hostname);
Query OK, 1 row affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
+----+--------+---------------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]>

blog2

■トランザクションの確認
NODE1)

root@localhost [GR_TEST]> start transaction;insert into T01(ID,MEMO) values(3,@@hostname);
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.00 sec)

root@localhost [GR_TEST]> commit;
Query OK, 0 rows affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | misc02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE2) 上記を実行中に同じテーブルに対してINSERTは問題ない。

root@localhost [GR_TEST]> start transaction;insert into T01(ID,MEMO) values(4,@@hostname);
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.00 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  4 | misc02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
3 rows in set (0.00 sec)

root@localhost [GR_TEST]> commit;
Query OK, 0 rows affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | misc02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

■ 同じ行を更新してみる(where ID=4)
NODE1)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | misc02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> start transaction;update T01 set MEMO = 'MISC02' where ID = 4;
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | MISC02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> commit;
Query OK, 0 rows affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | MISC02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> start transaction;update T01 set MEMO = 'misc02' where ID = 4;
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

root@localhost [GR_TEST]> commit;
Query OK, 0 rows affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | misc02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE2) NODE1のトランザクション中に実行。
※トランザクション発行の順番では無く、先にcommitした方が優先される。

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | misc02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> start transaction;update T01 set MEMO = 'MISC01' where ID = 4;
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | MISC01 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> select * from T01;
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
root@localhost [GR_TEST]> commit;
Query OK, 0 rows affected (0.00 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | MISC02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | MISC02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> start transaction;update T01 set MEMO = 'MISC01' where ID = 4;
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

root@localhost [GR_TEST]> commit;
ERROR 1180 (HY000): Got error 149 during COMMIT
root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-09-07 22:05:15 |
|  2 | misc02 | 2016-09-07 22:05:47 |
|  3 | misc01 | 2016-09-07 22:09:10 |
|  4 | misc02 | 2016-09-07 22:09:13 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

ログポジションベースのレプリケーションの場合は、レプリケーション障害発生時にスレーブでSKIP処理を容易に行う事ができましたが、GTIDモードの場合は以下のように空のトランザクションを実行して、エラー対応をする必要がありました。

mysqlslavetrxを利用しない場合の例

root@localhost [sakila]> stop slave;
Query OK, 0 rows affected (0.03 sec)

root@localhost [sakila]> SET GTID_NEXT = "3edaa0b8-3e39-11e4-9df1-080027f5bf08:54";
Query OK, 0 rows affected (0.00 sec)

root@localhost [sakila]> begin; commit;
Query OK, 0 rows affected (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

root@localhost [sakila]> SET GTID_NEXT = "3edaa0b8-3e39-11e4-9df1-080027f5bf08:55";
Query OK, 0 rows affected (0.00 sec)

root@localhost [sakila]> begin; commit;
Query OK, 0 rows affected (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

root@localhost [sakila]> SET GTID_NEXT = AUTOMATIC;
Query OK, 0 rows affected (0.00 sec)

root@localhost [sakila]> start slave;
Query OK, 0 rows affected (0.02 sec)

多くのトランザクションをSKIPする場合は、何回もコミットを実行して空のトランザクションを作成しないといけなかったので、非常に面倒な操作でしたが、MySQL Utilitiesの1.6以降で準備されているmysqlslavetrxを利用すると、まとめて1回で複数のGTIDをスキップ出来るようになっています。これまで以上にGTIDでの運用がし易くなっているので、活用頂ければと思います。

[root@misc02 scripts]# ./mysqlslavetrx --help
MySQL Utilities mysqlslavetrx version 1.6.4 
License type: GPLv2

1) 先ずは、レプリケーションに問題が発生していない状況を確認

■ マスターの状況

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc01     |
|  2 | mysqlslavetrx - 02 | misc01     |
|  3 | mysqlslavetrx - 03 | misc01     |
+----+--------------------+------------+
3 rows in set (0.01 sec)

root@localhost [REPLI]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                             |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.002412 |     3028 |              |                  | 2dde009f-d4dc-11e4-b437-0800279cea3c:1-222553 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)

root@localhost [REPLI]> 

■Slaveの状況

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc02     |
|  2 | mysqlslavetrx - 02 | misc02     |
|  3 | mysqlslavetrx - 03 | misc02     |
+----+--------------------+------------+
3 rows in set (0.00 sec)

root@localhost [REPLI]> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.56.113
                  Master_User: repl_user
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.002412
          Read_Master_Log_Pos: 3028
               Relay_Log_File: misc02-relay-bin.000133
                Relay_Log_Pos: 821
        Relay_Master_Log_File: mysql-bin.002412
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: REPLI
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 3028
              Relay_Log_Space: 3749
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: 2dde009f-d4dc-11e4-b437-0800279cea3c
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 2dde009f-d4dc-11e4-b437-0800279cea3c:216437-222553
            Executed_Gtid_Set: 2d0d3b32-4269-11e6-aca7-0800275fa837:1-7,
2dde009f-d4dc-11e4-b437-0800279cea3c:1-222553
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

root@localhost [REPLI]> 

■スレーブでデータ変更し、レプリケーションの不整合を発生させてみます。

スレーブでデータをINSERT

root@localhost [REPLI]> insert into T_Sample01(memo) values('Inserted @ Slave');
Query OK, 1 row affected (0.00 sec)

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc02     |
|  2 | mysqlslavetrx - 02 | misc02     |
|  3 | mysqlslavetrx - 03 | misc02     |
|  4 | Inserted @ Slave   | misc02     |
+----+--------------------+------------+
4 rows in set (0.00 sec)

root@localhost [REPLI]> 

マスターの状況

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc01     |
|  2 | mysqlslavetrx - 02 | misc01     |
|  3 | mysqlslavetrx - 03 | misc01     |
+----+--------------------+------------+
3 rows in set (0.00 sec)

root@localhost [REPLI]> 

■上記の状況でマスターでデータを変更

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc01     |
|  2 | mysqlslavetrx - 02 | misc01     |
|  3 | mysqlslavetrx - 03 | misc01     |
+----+--------------------+------------+
3 rows in set (0.00 sec)

root@localhost [REPLI]> insert into T_Sample01(memo) values('mysqlslavetrx - 04');
Query OK, 1 row affected (0.01 sec)

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc01     |
|  2 | mysqlslavetrx - 02 | misc01     |
|  3 | mysqlslavetrx - 03 | misc01     |
|  4 | mysqlslavetrx - 04 | misc01     |
+----+--------------------+------------+
4 rows in set (0.00 sec)

root@localhost [REPLI]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                             |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.002412 |     3395 |              |                  | 2dde009f-d4dc-11e4-b437-0800279cea3c:1-222554 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)

root@localhost [REPLI]> 

SLAVEではReplicationエラーになっている。(Duplicate Key)

root@localhost [REPLI]> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.56.113
                  Master_User: repl_user
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.002412
          Read_Master_Log_Pos: 3395
               Relay_Log_File: misc02-relay-bin.000133
                Relay_Log_Pos: 821
        Relay_Master_Log_File: mysql-bin.002412
             Slave_IO_Running: Yes
            Slave_SQL_Running: No
              Replicate_Do_DB: REPLI
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 1062
                   Last_Error: Could not execute Write_rows event on table REPLI.T_Sample01; Duplicate entry '4' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.002412, end_log_pos 3364
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 3028
              Relay_Log_Space: 4116
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 1062
               Last_SQL_Error: Could not execute Write_rows event on table REPLI.T_Sample01; Duplicate entry '4' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.002412, end_log_pos 3364
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: 2dde009f-d4dc-11e4-b437-0800279cea3c
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: 
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 160808 10:31:59
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 2dde009f-d4dc-11e4-b437-0800279cea3c:216437-222554
            Executed_Gtid_Set: 2d0d3b32-4269-11e6-aca7-0800275fa837:1-8,
2dde009f-d4dc-11e4-b437-0800279cea3c:1-222553
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

root@localhost [REPLI]> 

マスターでデータをもう1件変更(不要ですが、GTIDのSKIPを分かり易くする為)

root@localhost [REPLI]> insert into T_Sample01(memo) values('mysqlslavetrx - 05');
Query OK, 1 row affected (0.00 sec)

root@localhost [REPLI]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                             |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.002412 |     3762 |              |                  | 2dde009f-d4dc-11e4-b437-0800279cea3c:1-222555 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)

root@localhost [REPLI]> 

■Slave側ではリレーログの受信まで出来て、SQL(Applier)スレッドがINSERTに失敗しているので、
Duplicate entry ‘4’ for key ‘PRIMARY’の処理だけSKIPしてみる。

【対象】
Retrieved_Gtid_Set: 2dde009f-d4dc-11e4-b437-0800279cea3c:216437-222555
Executed_Gtid_Set: 2d0d3b32-4269-11e6-aca7-0800275fa837:1-8,
2dde009f-d4dc-11e4-b437-0800279cea3c:1-222553

 対象: 2dde009f-d4dc-11e4-b437-0800279cea3c:222554

mysqlslavetrxで対象のGTID(トランザクション)をSKIP実行

[root@misc02 scripts]# ./mysqlslavetrx --gtid-set=2dde009f-d4dc-11e4-b437-0800279cea3c:222554 --slaves=admin:password@misc02
WARNING: Using a password on the command line interface can be insecure.
#
# GTID set to be skipped for each server:
# - misc02@3306: 2dde009f-d4dc-11e4-b437-0800279cea3c:222554
#
# Injecting empty transactions for 'misc02:3306'...
#
#...done.
#
[root@misc02 scripts]# 

補足:複数GTIDをスキップする場合は以下のようにハイフンでつなぎます。

# Skip the specified GTID set (three transaction: 10, 11, 12) on two slaves.
$ mysqlslavetrx –gtid-set=ee2655ae-2e88-11e4-b7a3-606720440b68:10-12 –slaves=rpl:pass@host2:3306,rpl:pass@host3:3306

SKIP後のレプリケーションの状況を確認(レプリケーションが停止している状況が解消されている)
※リトライカウントがオーバーしている場合は、STOP SLAVE, START SLAVEを実行してみて下さい。


root@localhost [REPLI]> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.56.113
                  Master_User: repl_user
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.002412
          Read_Master_Log_Pos: 3762
               Relay_Log_File: misc02-relay-bin.000135
                Relay_Log_Pos: 454
        Relay_Master_Log_File: mysql-bin.002412
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: REPLI
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 3762
              Relay_Log_Space: 962
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: 2dde009f-d4dc-11e4-b437-0800279cea3c
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 2dde009f-d4dc-11e4-b437-0800279cea3c:216437-222555
            Executed_Gtid_Set: 2d0d3b32-4269-11e6-aca7-0800275fa837:1-8,
2dde009f-d4dc-11e4-b437-0800279cea3c:1-222555
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc02     |
|  2 | mysqlslavetrx - 02 | misc02     |
|  3 | mysqlslavetrx - 03 | misc02     |
|  4 | Inserted @ Slave   | misc02     |
|  5 | mysqlslavetrx - 05 | misc02     |
+----+--------------------+------------+
5 rows in set (0.00 sec)

root@localhost [REPLI]> 


以下、レプリケーション再開後のデータ修正例

■データ修正@SLAVE

root@localhost [REPLI]> set sql_log_bin = 0;
Query OK, 0 rows affected (0.00 sec)

root@localhost [REPLI]> update T_Sample01 set memo = 'mysqlslavetrx - 04' where id = 4;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

root@localhost [REPLI]> set sql_log_bin = 1;
Query OK, 0 rows affected (0.00 sec)

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc02     |
|  2 | mysqlslavetrx - 02 | misc02     |
|  3 | mysqlslavetrx - 03 | misc02     |
|  4 | mysqlslavetrx - 04 | misc02     |
|  5 | mysqlslavetrx - 05 | misc02     |
+----+--------------------+------------+
5 rows in set (0.00 sec)

root@localhost [REPLI]> 

■データ修正後の動作確認@マスター

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc01     |
|  2 | mysqlslavetrx - 02 | misc01     |
|  3 | mysqlslavetrx - 03 | misc01     |
|  4 | mysqlslavetrx - 04 | misc01     |
|  5 | mysqlslavetrx - 05 | misc01     |
+----+--------------------+------------+
5 rows in set (0.00 sec)

root@localhost [REPLI]> insert into T_Sample01(memo) values('mysqlslavetrx - 06');
Query OK, 1 row affected (0.00 sec)

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc01     |
|  2 | mysqlslavetrx - 02 | misc01     |
|  3 | mysqlslavetrx - 03 | misc01     |
|  4 | mysqlslavetrx - 04 | misc01     |
|  5 | mysqlslavetrx - 05 | misc01     |
|  6 | mysqlslavetrx - 06 | misc01     |
+----+--------------------+------------+
6 rows in set (0.01 sec)

root@localhost [REPLI]> 

■SLAVE

root@localhost [REPLI]> select *,@@hostname from T_Sample01;
+----+--------------------+------------+
| id | memo               | @@hostname |
+----+--------------------+------------+
|  1 | mysqlslavetrx - 01 | misc02     |
|  2 | mysqlslavetrx - 02 | misc02     |
|  3 | mysqlslavetrx - 03 | misc02     |
|  4 | mysqlslavetrx - 04 | misc02     |
|  5 | mysqlslavetrx - 05 | misc02     |
|  6 | mysqlslavetrx - 06 | misc02     |
+----+--------------------+------------+
6 rows in set (0.01 sec)

root@localhost [REPLI]> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.56.113
                  Master_User: repl_user
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.002412
          Read_Master_Log_Pos: 4129
               Relay_Log_File: misc02-relay-bin.000135
                Relay_Log_Pos: 821
        Relay_Master_Log_File: mysql-bin.002412
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: REPLI
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 4129
              Relay_Log_Space: 1329
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 1
                  Master_UUID: 2dde009f-d4dc-11e4-b437-0800279cea3c
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 2dde009f-d4dc-11e4-b437-0800279cea3c:216437-222556
            Executed_Gtid_Set: 2d0d3b32-4269-11e6-aca7-0800275fa837:1-8,
2dde009f-d4dc-11e4-b437-0800279cea3c:1-222556
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

root@localhost [REPLI]> 

詳細:
3.4.7 How do you fix errant transactions on the replication topology?


現状、まだLab版ですが、MySQL5.7の追加のプラグインとして、マルチマスタまたはアクティブ/アクティブレプリケーションをサポートする
同期レプリケーション型のグループレプリケーションが準備されています。まだ、Lab版という事もあり、機能追加やバグ対応などがまだまだ必要な段階ですが、LAB版→DR版→RC版→GA版と段々と安定してくると思いますので、次のLab版がリリースされたら是非検証環境で試してみて頂ければと思います。
マスターサーバーのHA対応やスレーブが多い環境で、マスターサーバーのレプリケーション負荷分散等に活用出来そうです。

Group Replication関連参考ブログを見て頂けると、基本的なインストール方法が書かれていますので試される場合は、此方を参考にして下さい。

http://mysqlhighavailability.com/getting-started-with-mysql-group-replication/

NODE1にてグループレプリケーション開始
※オプションファイルに書いておくことで、SETコマンドの実行は不要です。
※XCOMで通信する為のポートは、通常のMySQL PORT3306とは別にしてください。


root@localhost [mysql]> SET GLOBAL group_replication_group_name= "00000000-1111-2222-3333-123456789ABC";
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_bootstrap_group= 1;
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_local_address="192.168.56.101:13001";
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_peer_addresses= "192.168.56.101:13001,192.168.56.102:13001";
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_recovery_user='rpl_user';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_recovery_password='rpl_pass';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> START GROUP_REPLICATION;
Query OK, 0 rows affected (2.59 sec)

root@localhost [mysql]> SET GLOBAL group_replication_bootstrap_group= 0;
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SELECT * FROM performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: group_replication_applier
               GROUP_NAME: 00000000-1111-2222-3333-123456789ABC
              SOURCE_UUID: 00000000-1111-2222-3333-123456789ABC
                THREAD_ID: NULL
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 00000000-1111-2222-3333-123456789abc:1-4
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

root@localhost [mysql]> 


NODE2をGRに参加してみます。


root@localhost [mysql]> SET GLOBAL group_replication_group_name= "00000000-1111-2222-3333-123456789ABC";
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_local_address="192.168.56.102:13001";
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_peer_addresses= "192.168.56.101:13001,192.168.56.102:13001";
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_recovery_user='rpl_user';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SET GLOBAL group_replication_recovery_password='rpl_pass';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> START GROUP_REPLICATION;
Query OK, 0 rows affected (3.04 sec)

root@localhost [mysql]> SELECT * FROM performance_schema.replication_group_members\G
*************************** 1. row ***************************
CHANNEL_NAME: group_replication_applier
   MEMBER_ID: 29ea17bc-3848-11e6-9900-0800279ca844
 MEMBER_HOST: misc01
 MEMBER_PORT: 3306
MEMBER_STATE: ONLINE
*************************** 2. row ***************************
CHANNEL_NAME: group_replication_applier
   MEMBER_ID: 5b07d5d8-4057-11e6-a315-0800279cea3c
 MEMBER_HOST: misc02
 MEMBER_PORT: 3306
MEMBER_STATE: ONLINE
2 rows in set (0.01 sec)

root@localhost [mysql]> SELECT * FROM performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: group_replication_applier
               GROUP_NAME: 00000000-1111-2222-3333-123456789ABC
              SOURCE_UUID: 00000000-1111-2222-3333-123456789ABC
                THREAD_ID: NULL
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 00000000-1111-2222-3333-123456789abc:4
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

root@localhost [mysql]> 



group_members

MEMBER_STATEが共にONLINEになっているので、DDL、DMLを処理して同期されているか確認して見ます
先ずは、NODE1でデータベース、テーブルを作成してからデータを1件入れてみます。
NODE1で作成したオブジェクトやデータはNODE2でも確認出来ます。
また、同様にNODE2で入れたデータは、NODE1で確認する事が出来ます。


root@localhost [mysql]> CREATE DATABASE GR_TEST;
Query OK, 1 row affected (0.03 sec)

root@localhost [mysql]> use GR_TEST;
Database changed
root@localhost [GR_TEST]> CREATE TABLE GR_TEST.T01 (
    -> ID INT NOT NULL PRIMARY KEY,
    -> MEMO varchar(30) COLLATE utf8_bin NOT NULL DEFAULT ''
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
Query OK, 0 rows affected (0.07 sec)

root@localhost [GR_TEST]> INSERT INTO GR_TEST.T01(ID,MEMO) VALUES (1,@@hostname);
Query OK, 1 row affected (0.07 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
+----+--------+
1 row in set (0.01 sec)

root@localhost [GR_TEST]> 

NODE2でデータを確認してみます。


root@localhost [mysql]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| GR_TEST            |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

root@localhost [mysql]> use GR_TEST
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
+----+--------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> INSERT INTO GR_TEST.T01(ID,MEMO) VALUES (2,@@hostname);
Query OK, 1 row affected (0.04 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
|  2 | misc02 |
+----+--------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | ONLINE       |
| group_replication_applier | 5b07d5d8-4057-11e6-a315-0800279cea3c | misc02      |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: group_replication_applier
               GROUP_NAME: 00000000-1111-2222-3333-123456789ABC
              SOURCE_UUID: 00000000-1111-2222-3333-123456789ABC
                THREAD_ID: NULL
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 00000000-1111-2222-3333-123456789abc:4-7
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE2で入れたデータはNODE1でも確認出来ました。
これで、双方向にレプリケーションが張られている事が確認出来ました。


root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
|  2 | misc02 |
+----+--------+
2 rows in set (0.00 sec)


root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | ONLINE       |
| group_replication_applier | 5b07d5d8-4057-11e6-a315-0800279cea3c | misc02      |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: group_replication_applier
               GROUP_NAME: 00000000-1111-2222-3333-123456789ABC
              SOURCE_UUID: 00000000-1111-2222-3333-123456789ABC
                THREAD_ID: NULL
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 00000000-1111-2222-3333-123456789abc:4-7
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 

GTIDの状態を確認
自分で更新したデータに関しては、RECEIVED_TRANSACTION_SETには反映されないので、@@GLOBAL.GTID_EXECUTEDで何処まで適用されているか確認。


root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: group_replication_applier
               GROUP_NAME: 00000000-1111-2222-3333-123456789ABC
              SOURCE_UUID: 00000000-1111-2222-3333-123456789ABC
                THREAD_ID: NULL
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 00000000-1111-2222-3333-123456789abc:4-7:9
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

root@localhost [GR_TEST]> SELECT @@GLOBAL.GTID_EXECUTED;
+-------------------------------------------+
| @@GLOBAL.GTID_EXECUTED                    |
+-------------------------------------------+
| 00000000-1111-2222-3333-123456789abc:1-10 |
+-------------------------------------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE1とNODE2の間で、トランザクションの競合が発生した場合
(同時に同じデータを更新しようとした場合)

NODE1で先ずは、トランザクションを張って処理を実行してみます。そして、Commitを行う前に、NODE2で同じデータを更新処理してみます。
最初に処理を開始した、NODE1は問題無く処理出来てますが、NODE2のCommit処理はエラーで終了しています。

root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
|  2 | misc02 |
|  3 | misc01 |
|  4 | misc02 |
+----+--------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> start transaction;update T01 set MEMO = @@hostname where ID = 4;
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.03 sec)
Rows matched: 1  Changed: 1  Warnings: 0

root@localhost [GR_TEST]> commit;
Query OK, 0 rows affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
|  2 | misc02 |
|  3 | misc01 |
|  4 | misc01 |
+----+--------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE2は、ERROR 1180 (HY000): Got error 149 during COMMITでエラーになっています。

root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
|  2 | misc02 |
|  3 | misc01 |
|  4 | misc02 |
+----+--------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> start transaction;update T01 set MEMO = 'MISC02' where ID = 4;
Query OK, 0 rows affected (0.00 sec)

Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0

root@localhost [GR_TEST]> commit;
ERROR 1180 (HY000): Got error 149 during COMMIT
root@localhost [GR_TEST]> select * from T01;
+----+--------+
| ID | MEMO   |
+----+--------+
|  1 | misc01 |
|  2 | misc02 |
|  3 | misc01 |
|  4 | misc01 |
+----+--------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

group_replication_tran_conf

ONLINE DDLでALTERを実行した場合
結果としては、問題無くDDLも実行可能で伝搬されます。但し、現状のGRの仕様としては、オンラインスキーマ変更は推奨されていないので、
1)BINLOGをOFFにしてDDL実行 2) 他のノードも同様にBINLOGをOFFにしてDDL実行 3) 最後にアプリケーションを変更し変更を反映させるのが良さそうです。

NODE1にてDDLを実行して列を追加してみます。

root@localhost [GR_TEST]> ALTER TABLE T01 add column created_time datetime DEFAULT CURRENT_TIMESTAMP;
Query OK, 0 rows affected (0.12 sec)
Records: 0  Duplicates: 0  Warnings: 0

root@localhost [GR_TEST]> desc T01;
+--------------+-------------+------+-----+-------------------+-------+
| Field        | Type        | Null | Key | Default           | Extra |
+--------------+-------------+------+-----+-------------------+-------+
| ID           | int(11)     | NO   | PRI | NULL              |       |
| MEMO         | varchar(30) | NO   |     |                   |       |
| created_time | datetime    | YES  |     | CURRENT_TIMESTAMP |       |
+--------------+-------------+------+-----+-------------------+-------+
3 rows in set (0.03 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-07-17 15:40:29 |
|  2 | misc02 | 2016-07-17 15:40:29 |
|  3 | misc01 | 2016-07-17 15:40:29 |
|  4 | misc02 | 2016-07-17 15:40:29 |
+----+--------+---------------------+
4 rows in set (0.00 sec)

root@localhost [GR_TEST]> INSERT INTO GR_TEST.T01(ID,MEMO) VALUES (5,@@hostname);
Query OK, 1 row affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-07-17 15:40:29 |
|  2 | misc02 | 2016-07-17 15:40:29 |
|  3 | misc01 | 2016-07-17 15:40:29 |
|  4 | misc02 | 2016-07-17 15:40:29 |
|  5 | misc01 | 2016-07-17 15:42:33 |
+----+--------+---------------------+
5 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE2でNODE1で実行されたDDLの結果を確認して、NODE2からデータを追加してみます。

root@localhost [GR_TEST]> desc T01;
+--------------+-------------+------+-----+-------------------+-------+
| Field        | Type        | Null | Key | Default           | Extra |
+--------------+-------------+------+-----+-------------------+-------+
| ID           | int(11)     | NO   | PRI | NULL              |       |
| MEMO         | varchar(30) | NO   |     |                   |       |
| created_time | datetime    | YES  |     | CURRENT_TIMESTAMP |       |
+--------------+-------------+------+-----+-------------------+-------+
3 rows in set (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-07-17 15:40:29 |
|  2 | misc02 | 2016-07-17 15:40:29 |
|  3 | misc01 | 2016-07-17 15:40:29 |
|  4 | misc02 | 2016-07-17 15:40:29 |
|  5 | misc01 | 2016-07-17 15:42:33 |
+----+--------+---------------------+
5 rows in set (0.00 sec)

root@localhost [GR_TEST]> INSERT INTO GR_TEST.T01(ID,MEMO) VALUES (6,@@hostname);
Query OK, 1 row affected (0.01 sec)

root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-07-17 15:40:29 |
|  2 | misc02 | 2016-07-17 15:40:29 |
|  3 | misc01 | 2016-07-17 15:40:29 |
|  4 | misc02 | 2016-07-17 15:40:29 |
|  5 | misc01 | 2016-07-17 15:42:33 |
|  6 | misc02 | 2016-07-17 15:44:03 |
+----+--------+---------------------+
6 rows in set (0.00 sec)


root@localhost [GR_TEST]> SELECT @@GLOBAL.GTID_EXECUTED;
+-------------------------------------------+
| @@GLOBAL.GTID_EXECUTED                    |
+-------------------------------------------+
| 00000000-1111-2222-3333-123456789abc:1-15 |
+-------------------------------------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: group_replication_applier
               GROUP_NAME: 00000000-1111-2222-3333-123456789ABC
              SOURCE_UUID: 00000000-1111-2222-3333-123456789ABC
                THREAD_ID: NULL
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 00000000-1111-2222-3333-123456789abc:4-7:9:11:13-14
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | ONLINE       |
| group_replication_applier | 5b07d5d8-4057-11e6-a315-0800279cea3c | misc02      |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

NODE2でデータを入れたあとにNODE1の状況を確認してみます。
特に問題無く、データが反映されている事が確認出来ます。
レプリケーションもRAWベースなので特に時間関連の関数なども気にしなくて良いです。


root@localhost [GR_TEST]> select * from T01;
+----+--------+---------------------+
| ID | MEMO   | created_time        |
+----+--------+---------------------+
|  1 | misc01 | 2016-07-17 15:40:29 |
|  2 | misc02 | 2016-07-17 15:40:29 |
|  3 | misc01 | 2016-07-17 15:40:29 |
|  4 | misc02 | 2016-07-17 15:40:29 |
|  5 | misc01 | 2016-07-17 15:42:33 |
|  6 | misc02 | 2016-07-17 15:44:03 |
+----+--------+---------------------+
6 rows in set (0.01 sec)

root@localhost [GR_TEST]> SELECT @@GLOBAL.GTID_EXECUTED;
+-------------------------------------------+
| @@GLOBAL.GTID_EXECUTED                    |
+-------------------------------------------+
| 00000000-1111-2222-3333-123456789abc:1-15 |
+-------------------------------------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_connection_status\G
*************************** 1. row ***************************
             CHANNEL_NAME: group_replication_applier
               GROUP_NAME: 00000000-1111-2222-3333-123456789ABC
              SOURCE_UUID: 00000000-1111-2222-3333-123456789ABC
                THREAD_ID: NULL
            SERVICE_STATE: ON
COUNT_RECEIVED_HEARTBEATS: 0
 LAST_HEARTBEAT_TIMESTAMP: 0000-00-00 00:00:00
 RECEIVED_TRANSACTION_SET: 00000000-1111-2222-3333-123456789abc:1-4:8:10:12:15
        LAST_ERROR_NUMBER: 0
       LAST_ERROR_MESSAGE: 
     LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00
1 row in set (0.00 sec)

root@localhost [GR_TEST]> SELECT * FROM performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 29ea17bc-3848-11e6-9900-0800279ca844 | misc01      |        3306 | ONLINE       |
| group_replication_applier | 5b07d5d8-4057-11e6-a315-0800279cea3c | misc02      |        3306 | ONLINE       |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
2 rows in set (0.00 sec)

root@localhost [GR_TEST]> 

余談: Group Replicationは全てマスターなので、Show Slave Statusは不要ですね。


/*** NODE1 ***/
root@localhost [GR_TEST]> show slave status\G
Empty set (0.00 sec)

root@localhost [GR_TEST]> show master status;
+------------------+----------+--------------+------------------+-------------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                         |
+------------------+----------+--------------+------------------+-------------------------------------------+
| mysql-bin.000001 |     4391 |              |                  | 00000000-1111-2222-3333-123456789abc:1-15 |
+------------------+----------+--------------+------------------+-------------------------------------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 


/*** NODE2 ***/
root@localhost [GR_TEST]> show slave status\G
Empty set (0.02 sec)

root@localhost [GR_TEST]> show master status;
+------------------+----------+--------------+------------------+-------------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                         |
+------------------+----------+--------------+------------------+-------------------------------------------+
| mysql-bin.000001 |     4391 |              |                  | 00000000-1111-2222-3333-123456789abc:1-15 |
+------------------+----------+--------------+------------------+-------------------------------------------+
1 row in set (0.00 sec)

root@localhost [GR_TEST]> 

参考パラメータ


root@localhost [GR_TEST]> show global variables like '%group_repli%';
+---------------------------------------------------+-------------------------------------------+
| Variable_name                                     | Value                                     |
+---------------------------------------------------+-------------------------------------------+
| group_replication_allow_local_lower_version_join  | OFF                                       |
| group_replication_auto_increment_increment        | 7                                         |
| group_replication_bootstrap_group                 | OFF                                       |
| group_replication_components_stop_timeout         | 31536000                                  |
| group_replication_gcs_engine                      | xcom                                      |
| group_replication_group_name                      | 00000000-1111-2222-3333-123456789ABC      |
| group_replication_local_address                   | 192.168.56.101:13001                      |
| group_replication_peer_addresses                  | 192.168.56.101:13001,192.168.56.102:13001 |
| group_replication_pipeline_type_var               | STANDARD                                  |
| group_replication_recovery_complete_at            | TRANSACTIONS_CERTIFIED                    |
| group_replication_recovery_password               |                                           |
| group_replication_recovery_reconnect_interval     | 120                                       |
| group_replication_recovery_retry_count            | 2                                         |
| group_replication_recovery_ssl_ca                 |                                           |
| group_replication_recovery_ssl_capath             |                                           |
| group_replication_recovery_ssl_cert               |                                           |
| group_replication_recovery_ssl_cipher             |                                           |
| group_replication_recovery_ssl_crl                |                                           |
| group_replication_recovery_ssl_crlpath            |                                           |
| group_replication_recovery_ssl_key                |                                           |
| group_replication_recovery_ssl_verify_server_cert | OFF                                       |
| group_replication_recovery_use_ssl                | OFF                                       |
| group_replication_recovery_user                   | rpl_user                                  |
| group_replication_start_on_boot                   | OFF                                       |
+---------------------------------------------------+-------------------------------------------+
24 rows in set (0.01 sec)

root@localhost [GR_TEST]> 

【参考】

LABサイト
http://labs.mysql.com/

Group Replication関連参考ブログ
http://mysqlhighavailability.com/getting-started-with-mysql-group-replication/

MySQLセミナー資料
http://downloads.mysql.com/presentations/20160510_06_MySQL_57_ReplicationEnhancements.pdf

Auto Incrementの値
http://mysqlhighavailability.com/mysql-group-replication-auto-increment-configuration-handling/


MySQL5.7.13以降における、Enterprise Audit機能の改善

MySQL5.7.13以降で、全てのユーザー若しくは、特定ユーザーが特定のテーブルに対して行った、read,insert,update,delete処理のみを監査出来るようになりました。
フィルター作成はJSONフォーマットで定義するようです。この機能は、以前から待ち望んでいたので、嬉しい機能の一つです。

【検証バージョン】5.7.13-enterprise-commercial-advanced-log

フィルタリング詳細:
https://dev.mysql.com/doc/refman/5.7/en/audit-log-filtering.html

例) こちらは、confidentialテーブルに対してselectしたSQLだけ監査するようにフィルターしてある場合の監査ログです。
【フィルター】

root@localhost [mysql]> select * from audit_log_filter;
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
| NAME             | FILTER                                                                                                                                            |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
| log_all          | {"filter": {"log": true}}                                                                                                                         |
| log_confidential | {"filter": {"class": {"name": "table_access", "event": {"log": {"field": {"name": "table_name.str", "value": "confidential"}}, "name": "read"}}}} |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+

【監査ログ】
audit

1) 以下のように書かれていたので、既にインストール済みバージョンをアンインストールしました。
If the audit_log plugin is already installed from a version of MySQL before 5.7.13,
uninstall it using this statement and then restart the server before installing the current version:

■ オプションファイルもコメントアウト


[mysqld]
#plugin-load=audit_log.so
#audit-log=FORCE_PLUS_PERMANENT

■ Uninstall作業

root@localhost [(none)]> UNINSTALL PLUGIN audit_log;
Query OK, 0 rows affected, 1 warning (0.00 sec)

root@localhost [(none)]> show warnings;
+---------+------+----------------------------------------------------+
| Level   | Code | Message                                            |
+---------+------+----------------------------------------------------+
| Warning | 1620 | Plugin is busy and will be uninstalled on shutdown |
+---------+------+----------------------------------------------------+
1 row in set (0.00 sec)

root@localhost [(none)]> exit
Bye


[root@misc02 admin]# /etc/init.d/mysql.server restart

■ ロードされていない事を確認してインストール準備完了

root@localhost [information_Schema]> select PLUGIN_NAME,PLUGIN_STATUS,PLUGIN_TYPE,LOAD_OPTION from FROM INFORMATION_SCHEMA.PLUGINS where PLUGIN_NAME like 'audit%';
Empty set (0.00 sec)

root@localhost [information_Schema]> 

■ログ
参考) 5.7.13のログを確認したら、UNINSTALL前は以下のようなログが出ていました。

[root@misc01 admin]# cat /usr/local/mysql/data/error.log | grep audit
2016-06-15T21:26:01.568228+09:00 0 [Warning] Plugin audit_log reported: 'Audit Log plugin supports a filtering, which has not been installed yet. Audit Log plugin will run in the legacy mode, which will be disabled in the next release.'
2016-06-15T21:34:11.395386+09:00 4 [Note] Shutting down plugin 'audit_log'
[root@misc01 admin]# 

2)スクリプトを利用してインストールします。(これまで、Audit Pluginを利用した事が無ければここから設定開始)

スクリプトの内容を確認すると2つのテーブルと5つのファンクション,そして、AUDIT PLUGINのインストールを行っています。テーブルはInnoDBに進もうとしているのに、なぜかMyISAMを利用していますが。。。あとユーザーのサイズ制限がなぜかVARCHAAR(16)
こちらは、後で問い合わせしてみたいと思います。
———————————————————————————-
audit_log_filter_linux_install.sqlの内容
———————————————————————————-


USE mysql;

CREATE TABLE IF NOT EXISTS audit_log_filter(NAME VARCHAR(64) BINARY NOT NULL PRIMARY KEY, FILTER JSON NOT NULL) engine= MyISAM;
CREATE TABLE IF NOT EXISTS audit_log_user(USER VARCHAR(16) BINARY NOT NULL, HOST VARCHAR(60) BINARY NOT NULL, FILTERNAME VARCHAR(64) BINARY NOT NULL, PRIMARY KEY (USER, HOST), FOREIGN KEY (FILTERNAME) REFERENCES mysql.audit_log_filter(NAME)) engine= MyISAM;

INSTALL PLUGIN audit_log SONAME 'audit_log.so';

CREATE FUNCTION audit_log_filter_set_filter RETURNS STRING SONAME 'audit_log.so';
CREATE FUNCTION audit_log_filter_remove_filter RETURNS STRING SONAME 'audit_log.so';
CREATE FUNCTION audit_log_filter_set_user RETURNS STRING SONAME 'audit_log.so';
CREATE FUNCTION audit_log_filter_remove_user RETURNS STRING SONAME 'audit_log.so';
CREATE FUNCTION audit_log_filter_flush RETURNS STRING SONAME 'audit_log.so';

SELECT audit_log_filter_flush() AS 'Result';

Installの実行
インストールスクリプトはSHAREフォルダーにあります。(WindowsとLinuxで別々なので注意して下さい)


[root@misc01 admin]# mysql -u root -p < /usr/local/mysql/share/audit_log_filter_linux_install.sql 
Enter password: 
Result
OK
[root@misc01 admin]# 


[root@misc01 admin]# mysql -u root -p -e "select PLUGIN_NAME,PLUGIN_STATUS,PLUGIN_TYPE,LOAD_OPTION FROM INFORMATION_SCHEMA.PLUGINS where PLUGIN_NAME like 'audit%'"
Enter password: 
+-------------+---------------+-------------+-------------+
| PLUGIN_NAME | PLUGIN_STATUS | PLUGIN_TYPE | LOAD_OPTION |
+-------------+---------------+-------------+-------------+
| audit_log   | ACTIVE        | AUDIT       | ON          |
+-------------+---------------+-------------+-------------+
[root@misc01 admin]# 




root@localhost [mysql]> show tables from mysql like 'audit%';
+--------------------------+
| Tables_in_mysql (audit%) |
+--------------------------+
| audit_log_filter         |
| audit_log_user           |
+--------------------------+
2 rows in set (0.00 sec)

root@localhost [mysql]> 

ここで、監査の初期設定は完了です。
これから、フィルター設定を幾つか入れて確認してみます。

3) 監査用のユーザー作成、ルール作成、ルール適用して検証してみます


root@localhost [mysql]> CREATE USER 'audit_target';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> GRANT ALL ON *.* TO 'audit_target';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SELECT user, host FROM mysql.user WHERE user = 'audit_target';
+----------------+------+
| user           | host |
+----------------+------+
| audit_target   | %    |
+----------------+------+
1 row in set (0.00 sec)

root@localhost [mysql]> select * from audit_log_user;
Empty set (0.00 sec)

root@localhost [mysql]> select * from audit_log_filter;
Empty set (0.00 sec)

root@localhost [mysql]> SELECT audit_log_filter_set_filter('log_all', '{ "filter": { "log": true } }') AS 'Result';
+--------+
| Result |
+--------+
| OK     |
+--------+
1 row in set (0.00 sec)

root@localhost [mysql]> select * from audit_log_filter;
+-------------+---------------------------+
| NAME        | FILTER                    |
+-------------+---------------------------+
| log_all     | {"filter": {"log": true}} |
+-------------+---------------------------+
1 row in set (0.00 sec)

root@localhost [mysql]> SELECT audit_log_filter_set_user('audit_target@%', 'log_all') AS 'Result';
+--------+
| Result |
+--------+
| OK     |
+--------+
1 row in set (0.00 sec)

root@localhost [mysql]> select * from audit_log_user;
+----------------+------+-------------+
| USER           | HOST | FILTERNAME  |
+----------------+------+-------------+
| audit_target   | %    | log_all     |
+----------------+------+-------------+
1 row in set (0.00 sec)

root@localhost [mysql]> 

4) 別ホストから対象アカウントを利用してアクセスしてみます。


[root@misc02 admin]# mysql -h 192.168.56.113 -u audit_target
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 12
Server version: 5.7.13-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

audit_target@192.168.56.113 [(none)]> CREATE DATABASE audit_log_test_db;
Query OK, 1 row affected (0.05 sec)

audit_target@192.168.56.113 [(none)]> USE audit_log_test_db;
Database changed

audit_target@192.168.56.113 [audit_log_test_db]> CREATE TABLE confidential (memo varchar(100));
Query OK, 0 rows affected (0.02 sec)

audit_target@192.168.56.113 [audit_log_test_db]> INSERT INTO audit_log_test_table VALUES(1);
Query OK, 1 row affected (0.02 sec)

audit_target@192.168.56.113 [audit_log_test_db]> exit
Bye
[root@misc02 admin]# 

■上記のログがAuditログに記録されませんでした。
検証したところ、USER()を見ているようでした。

注意

current_user()を認識していないのは、Bugとの回答を頂きました。
既に修正済みで、次のメンテナンスリリースでFIXされるとの事でした。
なので、MySQL5.7.14を待ちたいと思います。

FIX後は以下の様に’%’でホストを指定している場合、user()毎に作成する必要は有りません。

また、JSONで定義を作成し有効にした場合に、mysqlクライアントからの接続出来ますが、
workbenchから接続出来なくなる不具合があるようです。Bug Reportを上げたので、5.7.13で利用される場合は制限がある事をご理解下さい。
http://bugs.mysql.com/bug.php?id=81897

上記修正されました。
Audit log filtering against the user was performing comparisons against USER(), not CURRENT_USER(). (Bug #23344762)
https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-14.html


audit_target@192.168.56.113 [(none)]> select user(),current_user();
+-----------------------+------------------+
| user()                | current_user()   |
+-----------------------+------------------+
| audit_target@misc02   | audit_target@%   |
+-----------------------+------------------+
1 row in set (0.00 sec)

以下のように対象HOSTを追加しました。
追加したところ、audit_targetのAuditログが記録されている事が確認出来ました。

(例)
SELECT audit_log_filter_set_user(‘audit_target@misc02’, ‘log_all’) AS ‘Result’;


root@localhost [mysql]> select * from audit_log_user;select * from audit_log_filter;
+--------------+----------------+------------+
| USER         | HOST           | FILTERNAME |
+--------------+----------------+------------+
| audit_target | localhost      | log_all    |
| audit_target | %              | log_all    |
| audit_target | 192.168.56.109 | log_all    |
| audit_target | misc02         | log_all    |
+--------------+----------------+------------+
4 rows in set (0.00 sec)

root@localhost [mysql]> 

5) 基本的な機能は確認出来たので、ユーザー、テーブル、DMLの種類によるフィルタリングを確認してみます。
※ 全ての処理はもちろん取得し記録出来るので、特定のオブジェクトに対しての特定の処理のみを確認しています。

ここではユーザーを作成して、confidentialテーブルへのREAD(select)のみをログに記録する設定をしています。
READのみなので、INSERTはここでは取得していません。


root@localhost [mysql]> select * from audit_log_user;select * from audit_log_filter;
+--------------+----------------+------------+
| USER         | HOST           | FILTERNAME |
+--------------+----------------+------------+
| audit_target | localhost      | log_all    |
| audit_target | %              | log_all    |
| audit_target | 192.168.56.109 | log_all    |
| audit_target | misc02         | log_all    |
+--------------+----------------+------------+
4 rows in set (0.00 sec)

root@localhost [mysql]> SELECT audit_log_filter_set_filter('log_confidential', '{ "filter": { "class": { "name": "table_access","event": { "name": "read","log": { "field": { "name": "table_name.str","value": "confidential" } } } } } }');
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| audit_log_filter_set_filter('log_confidential', '{ "filter": { "class": { "name": "table_access","event": { "name": "read","log": { "field": { "name": "table_name.str","value": "confidential" } } } } } }') |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OK                                                                                                                                                                                                              |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

audit_target@localhost [mysql]> 


root@localhost [mysql]> CREATE USER confidential@'%';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> alter user confidential@'%' identified by 'password';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> GRANT ALL ON *.* TO 'confidential'@'%';
Query OK, 0 rows affected (0.00 sec)

root@localhost [mysql]> SELECT audit_log_filter_set_user('confidential@%','log_confidential');
+----------------------------------------------------------------+
| audit_log_filter_set_user('confidential@%','log_confidential') |
+----------------------------------------------------------------+
| OK                                                             |
+----------------------------------------------------------------+
1 row in set (0.00 sec)

root@localhost [mysql]> SELECT audit_log_filter_set_user('confidential@localhost','log_confidential');
+------------------------------------------------------------------------+
| audit_log_filter_set_user('confidential@localhost','log_confidential') |
+------------------------------------------------------------------------+
| OK                                                                     |
+------------------------------------------------------------------------+
1 row in set (0.00 sec)

root@localhost [mysql]>  SELECT audit_log_filter_set_user('confidential@192.168.56.109','log_confidential');
+-----------------------------------------------------------------------------+
| audit_log_filter_set_user('confidential@192.168.56.109','log_confidential') |
+-----------------------------------------------------------------------------+
| OK                                                                          |
+-----------------------------------------------------------------------------+
1 row in set (0.00 sec)

root@localhost [mysql]> SELECT audit_log_filter_set_user('confidential@misc02','log_confidential');
+---------------------------------------------------------------------+
| audit_log_filter_set_user('confidential@misc02','log_confidential') |
+---------------------------------------------------------------------+
| OK                                                                  |
+---------------------------------------------------------------------+
1 row in set (0.00 sec)

root@localhost [mysql]> select * from audit_log_filter;
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
| NAME             | FILTER                                                                                                                                            |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
| log_all          | {"filter": {"log": true}}                                                                                                                         |
| log_confidential | {"filter": {"class": {"name": "table_access", "event": {"log": {"field": {"name": "table_name.str", "value": "confidential"}}, "name": "read"}}}} |
+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

root@localhost [mysql]> select * from audit_log_user;
+--------------+----------------+------------------+
| USER         | HOST           | FILTERNAME       |
+--------------+----------------+------------------+
| audit_target | localhost      | log_all          |
| audit_target | %              | log_all          |
| audit_target | 192.168.56.109 | log_all          |
| audit_target | misc02         | log_all          |
| confidential | %              | log_confidential |
| confidential | localhost      | log_confidential |
| confidential | 192.168.56.109 | log_confidential |
| confidential | misc02         | log_confidential |
+--------------+----------------+------------------+
8 rows in set (0.00 sec)

root@localhost [mysql]> 

6) 上記コマンドを実行して、別ホストからアクセスしてログを確認してみました。
Select -> Insert -> Selectの順番でログを確認しています。


confidential@192.168.56.113 [Audit]> select * from confidential;
Empty set (0.00 sec)

confidential@192.168.56.113 [Audit]> insert into confidential(memo) values('秘密の情報');
Query OK, 1 row affected (0.01 sec)

confidential@192.168.56.113 [Audit]> select * from confidential;
+-----------------+
| memo            |
+-----------------+
| 秘密の情報      |
+-----------------+
1 row in set (0.01 sec)

confidential@192.168.56.113 [Audit]> 

audit.logの中身を確認してみると。
きちんと設定したテーブルに対しての参照処理のみ記録されている事が確認出来ました。


 <AUDIT_RECORD>>
  <TIMESTAMP>2016-06-16T03:09:53 UTC</TIMESTAMP>
  <RECORD_ID>87435_2016-06-16T02:35:04</RECORD_ID>
  <NAME>TableRead</NAME>
  <CONNECTION_ID>6</CONNECTION_ID>
  <USER>confidential[confidential] @ misc02 [192.168.56.109]</USER>
  <OS_LOGIN/>
  <HOST>misc02</HOST>
  <IP>192.168.56.109</IP>
  <COMMAND_CLASS>select</COMMAND_CLASS>
  <SQLTEXT>select * from confidential</SQLTEXT>
  <DB>Audit</DB>
  <TABLE>confidential</TABLE>
 </AUDIT_RECORD>
 <AUDIT_RECORD>
  <TIMESTAMP>2016-06-16T03:10:12 UTC</TIMESTAMP>
  <RECORD_ID>87436_2016-06-16T02:35:04</RECORD_ID>
  <NAME>TableRead</NAME>
  <CONNECTION_ID>6</CONNECTION_ID>
  <USER>confidential[confidential] @ misc02 [192.168.56.109]</USER>
  <OS_LOGIN/>
  <HOST>misc02</HOST>
  <IP>192.168.56.109</IP>
  <COMMAND_CLASS>select</COMMAND_CLASS>
  <SQLTEXT>select * from confidential</SQLTEXT>
  <DB>Audit</DB>
  <TABLE>confidential</TABLE>
 </AUDIT_RECORD>

fixed

これまでより、断然監査がし易くなっているので是非検証してみて下さい。
トライアル(試用版)ダウンロード: https://www-jp.mysql.com/trials/

Enterprise Monitorも合わせて検証するとAudit Logの状況がリアルタイムで可視化出来ます。(閾値を超えたら、メールかSNMPで管理者に連絡)
index

indexlog

参考)
https://dev.mysql.com/doc/refman/5.7/en/audit-log-installation.html
http://mysqlserverteam.com/mysql-5-7-new-audit-log-filtering-feature-part-1/