运维工作中你遇到过哪些神奇事件？

GreatSQL社区»论坛问答 › GreatSQL › GreatSQL

[已解决] GreatSQL 8.0.32-25 MGR集群使用正常，但频繁报错[ERROR] [MY-013780]

163 2 2024-4-16 14:46

本帖最后由景玉于 2024-4-16 14:55 编辑

GreatSQL 版本：mysql  Ver 8.0.32-25 for Linux on x86_64 (GreatSQL, Release 25, Revision 79f57097e3f-gmssl)

MGR集群搭建方式：GreatSQL Shell 8.0.32-25 构建MGR集群
集群成员状态：

(Tue Apr 16 14:35:21 2024)[root@GreatSQL][mysql_innodb_cluster_metadata]>select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME             | MEMBER_ID                         | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | 4cbc4ba9-f158-11ee-b8ac-08002751cf3a | 192.168.56.81 |       3306 | ONLINE    | PRIMARY    | 8.0.32       | MySQL                   |
| group_replication_applier | 4fdcda4f-f158-11ee-b7df-08002732cb3d | 192.168.56.82 |       3306 | ONLINE    | SECONDARY | 8.0.32       | MySQL                   |
| group_replication_applier | 52b6d17d-f158-11ee-b83f-080027ca9bf3 | 192.168.56.83 |       3306 | ONLINE    | SECONDARY | 8.0.32       | MySQL                   |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+

该MGR集群做过节点断网测试，之前添加的仲裁节点已经移除。VIP功能已禁用。
192.168.56.81:3306、192.168.56.82:3306两个实力都断网事件较长，都做过rejoinInstance)() 操作。

192.168.56.81:3306、192.168.56.82:3306两个实例频繁报错信息(192.168.56.83:3306无报错信息)：
.......
2024-04-16T14:44:55.726940+08:00 0 [ERROR] [MY-013780] [Repl] Plugin group_replication reported: 'Failed to establish MySQL client connection in Group Replication. Error establishing connection. Please refer to the manual to make sure that you configured Group Replication properly to work with MySQL Protocol connections.'
2024-04-16T14:44:56.732852+08:00 0 [ERROR] [MY-013780] [Repl] Plugin group_replication reported: 'Failed to establish MySQL client connection in Group Replication. Error establishing connection. Please refer to the manual to make sure that you configured Group Replication properly to work with MySQL Protocol connections.'
2024-04-16T14:44:57.748939+08:00 0 [ERROR] [MY-013780] [Repl] Plugin group_replication reported: 'Failed to establish MySQL client connection in Group Replication. Error establishing connection. Please refer to the manual to make sure that you configured Group Replication properly to work with MySQL Protocol connections.'
......

192.168.56.81:3306  实例MGR配置参数：

(Tue Apr 16 14:47:18 2024)[root@GreatSQL][sys]>show global variables like 'group_rep%';
+--------------------------------------------------------------+---------------------------------------+
| Variable_name                                              | Value                               |
+--------------------------------------------------------------+---------------------------------------+
| group_replication_advertise_recovery_endpoints             | DEFAULT                            |
| group_replication_allow_local_lower_version_join          | OFF                                  |
| group_replication_applier_batch_size_threshold             | 100000                               |
| group_replication_arbitrator                               | OFF                                  |
| group_replication_async_auto_failover_channel_read_only_mode | OFF                                  |
| group_replication_auto_evict_timeout                      | 0                                  |
| group_replication_auto_increment_increment                | 7                                  |
| group_replication_autorejoin_tries                         | 288                                  |
| group_replication_bootstrap_group                         | OFF                                  |
| group_replication_broadcast_gtid_executed_period          | 1000                               |
| group_replication_clone_threshold                         | 9223372036854775807                |
| group_replication_communication_debug_options             | GCS_DEBUG_NONE                      |
| group_replication_communication_flp_timeout                | 5                                  |
| group_replication_communication_max_message_size          | 10485760                            |
| group_replication_communication_stack                      | MYSQL                               |
| group_replication_components_stop_timeout                   | 300                                  |
| group_replication_compression_threshold                   | 1000000                            |
| group_replication_consistency                               | EVENTUAL                            |
| group_replication_donor_threshold                         | 9223372036854775807                |
| group_replication_enforce_update_everywhere_checks          | OFF                                  |
| group_replication_exit_state_action                         | READ_ONLY                            |
| group_replication_flow_control_applier_threshold          | 25000                               |
| group_replication_flow_control_certifier_threshold          | 25000                               |
| group_replication_flow_control_hold_percent                | 10                                  |
| group_replication_flow_control_max_quota                   | 0                                  |
| group_replication_flow_control_max_wait_time                | 3600                               |
| group_replication_flow_control_member_quota_percent       | 0                                  |
| group_replication_flow_control_min_quota                   | 0                                  |
| group_replication_flow_control_min_recovery_quota          | 0                                  |
| group_replication_flow_control_mode                         | DISABLED                            |
| group_replication_flow_control_period                      | 1                                  |
| group_replication_flow_control_release_percent             | 50                                  |
| group_replication_flow_control_replay_lag_behind          | 600                                  |
| group_replication_force_members                            |                                     |
| group_replication_group_name                               | f2a0dd86-f159-11ee-ac63-08002751cf3a  |
| group_replication_group_seeds                               | 192.168.56.82:3306,192.168.56.83:3306 |
| group_replication_gtid_assignment_block_size                | 1000000                            |
| group_replication_ip_allowlist                            | AUTOMATIC                            |
| group_replication_ip_whitelist                            | AUTOMATIC                            |
| group_replication_local_address                            | 192.168.56.81:3306                   |
| group_replication_majority_after_mode                      | ON                                  |
| group_replication_member_expel_timeout                      | 5                                  |
| group_replication_member_weight                            | 50                                  |
| group_replication_message_cache_size                      | 1073741824                         |
| group_replication_paxos_single_leader                      | OFF                                  |
| group_replication_poll_spin_loops                         | 0                                  |
| group_replication_primary_election_mode                   | GTID_FIRST                         |
| group_replication_recovery_complete_at                      | TRANSACTIONS_APPLIED                |
| group_replication_recovery_compression_algorithms          | uncompressed                         |
| group_replication_recovery_get_public_key                   | ON                                  |
| group_replication_recovery_public_key_path                |                                     |
| group_replication_recovery_reconnect_interval             | 60                                  |
| group_replication_recovery_retry_count                      | 10                                  |
| group_replication_recovery_ssl_ca                         |                                     |
| group_replication_recovery_ssl_capath                      |                                     |
| group_replication_recovery_ssl_cert                         |                                     |
| group_replication_recovery_ssl_cipher                      |                                     |
| group_replication_recovery_ssl_crl                         |                                     |
| group_replication_recovery_ssl_crlpath                      |                                     |
| group_replication_recovery_ssl_key                         |                                     |
| group_replication_recovery_ssl_verify_server_cert          | OFF                                  |
| group_replication_recovery_tls_ciphersuites                |                                     |
| group_replication_recovery_tls_version                      | TLSv1.2,TLSv1.3                      |
| group_replication_recovery_use_ssl                         | ON                                  |
| group_replication_recovery_zstd_compression_level          | 3                                  |
| group_replication_request_time_threshold                   | 100                                  |
| group_replication_single_primary_fast_mode                | 0                                  |
| group_replication_single_primary_mode                      | ON                                  |
| group_replication_ssl_mode                                  | REQUIRED                            |
| group_replication_start_on_boot                            | ON                                  |
| group_replication_tls_source                               | MYSQL_MAIN                         |
| group_replication_transaction_size_limit                   | 150000000                            |
| group_replication_unreachable_majority_timeout             | 0                                  |
| group_replication_view_change_uuid                         | f2a0eb31-f159-11ee-ac63-08002751cf3a  |
| group_replication_xcom_cache_mode                         | 2                                  |
| group_replication_zone_id                                  | 0                                  |
| group_replication_zone_id_sync_mode                         | ON                                  |
+--------------------------------------------------------------+---------------------------------------+
77 rows in set (0.00 sec)

192.168.56.82:3306  实例MGR配置参数：

(Tue Apr 16 14:14:45 2024)[root@GreatSQL][(none)]>show global variables like 'group_rep%';
+--------------------------------------------------------------+---------------------------------------+
| Variable_name                                              | Value                               |
+--------------------------------------------------------------+---------------------------------------+
| group_replication_advertise_recovery_endpoints             | DEFAULT                            |
| group_replication_allow_local_lower_version_join          | OFF                                  |
| group_replication_applier_batch_size_threshold             | 100000                               |
| group_replication_arbitrator                               | OFF                                  |
| group_replication_async_auto_failover_channel_read_only_mode | OFF                                  |
| group_replication_auto_evict_timeout                      | 0                                  |
| group_replication_auto_increment_increment                | 7                                  |
| group_replication_autorejoin_tries                         | 288                                  |
| group_replication_bootstrap_group                         | OFF                                  |
| group_replication_broadcast_gtid_executed_period          | 1000                               |
| group_replication_clone_threshold                         | 9223372036854775807                |
| group_replication_communication_debug_options             | GCS_DEBUG_NONE                      |
| group_replication_communication_flp_timeout                | 5                                  |
| group_replication_communication_max_message_size          | 10485760                            |
| group_replication_communication_stack                      | MYSQL                               |
| group_replication_components_stop_timeout                   | 300                                  |
| group_replication_compression_threshold                   | 1000000                            |
| group_replication_consistency                               | EVENTUAL                            |
| group_replication_donor_threshold                         | 9223372036854775807                |
| group_replication_enforce_update_everywhere_checks          | OFF                                  |
| group_replication_exit_state_action                         | READ_ONLY                            |
| group_replication_flow_control_applier_threshold          | 25000                               |
| group_replication_flow_control_certifier_threshold          | 25000                               |
| group_replication_flow_control_hold_percent                | 10                                  |
| group_replication_flow_control_max_quota                   | 0                                  |
| group_replication_flow_control_max_wait_time                | 3600                               |
| group_replication_flow_control_member_quota_percent       | 0                                  |
| group_replication_flow_control_min_quota                   | 0                                  |
| group_replication_flow_control_min_recovery_quota          | 0                                  |
| group_replication_flow_control_mode                         | DISABLED                            |
| group_replication_flow_control_period                      | 1                                  |
| group_replication_flow_control_release_percent             | 50                                  |
| group_replication_flow_control_replay_lag_behind          | 600                                  |
| group_replication_force_members                            |                                     |
| group_replication_group_name                               | f2a0dd86-f159-11ee-ac63-08002751cf3a  |
| group_replication_group_seeds                               | 192.168.56.81:3306,192.168.56.83:3306 |
| group_replication_gtid_assignment_block_size                | 1000000                            |
| group_replication_ip_allowlist                            | AUTOMATIC                            |
| group_replication_ip_whitelist                            | AUTOMATIC                            |
| group_replication_local_address                            | 192.168.56.82:3306                   |
| group_replication_majority_after_mode                      | ON                                  |
| group_replication_member_expel_timeout                      | 5                                  |
| group_replication_member_weight                            | 50                                  |
| group_replication_message_cache_size                      | 1073741824                         |
| group_replication_paxos_single_leader                      | OFF                                  |
| group_replication_poll_spin_loops                         | 0                                  |
| group_replication_primary_election_mode                   | GTID_FIRST                         |
| group_replication_recovery_complete_at                      | TRANSACTIONS_APPLIED                |
| group_replication_recovery_compression_algorithms          | uncompressed                         |
| group_replication_recovery_get_public_key                   | ON                                  |
| group_replication_recovery_public_key_path                |                                     |
| group_replication_recovery_reconnect_interval             | 60                                  |
| group_replication_recovery_retry_count                      | 10                                  |
| group_replication_recovery_ssl_ca                         |                                     |
| group_replication_recovery_ssl_capath                      |                                     |
| group_replication_recovery_ssl_cert                         |                                     |
| group_replication_recovery_ssl_cipher                      |                                     |
| group_replication_recovery_ssl_crl                         |                                     |
| group_replication_recovery_ssl_crlpath                      |                                     |
| group_replication_recovery_ssl_key                         |                                     |
| group_replication_recovery_ssl_verify_server_cert          | OFF                                  |
| group_replication_recovery_tls_ciphersuites                |                                     |
| group_replication_recovery_tls_version                      | TLSv1.2,TLSv1.3                      |
| group_replication_recovery_use_ssl                         | ON                                  |
| group_replication_recovery_zstd_compression_level          | 3                                  |
| group_replication_request_time_threshold                   | 100                                  |
| group_replication_single_primary_fast_mode                | 0                                  |
| group_replication_single_primary_mode                      | ON                                  |
| group_replication_ssl_mode                                  | REQUIRED                            |
| group_replication_start_on_boot                            | ON                                  |
| group_replication_tls_source                               | MYSQL_MAIN                         |
| group_replication_transaction_size_limit                   | 150000000                            |
| group_replication_unreachable_majority_timeout             | 0                                  |
| group_replication_view_change_uuid                         | f2a0eb31-f159-11ee-ac63-08002751cf3a  |
| group_replication_xcom_cache_mode                         | 2                                  |
| group_replication_zone_id                                  | 0                                  |
| group_replication_zone_id_sync_mode                         | ON                                  |
+--------------------------------------------------------------+---------------------------------------+
77 rows in set (0.01 sec)

192.168.56.83:3306  实例MGR配置参数：

(Tue Apr 16 14:16:19 2024)[root@GreatSQL][(none)]>show global variables like 'group_rep%';
+--------------------------------------------------------------+---------------------------------------+
| Variable_name                                              | Value                               |
+--------------------------------------------------------------+---------------------------------------+
| group_replication_advertise_recovery_endpoints             | DEFAULT                            |
| group_replication_allow_local_lower_version_join          | OFF                                  |
| group_replication_applier_batch_size_threshold             | 100000                               |
| group_replication_arbitrator                               | OFF                                  |
| group_replication_async_auto_failover_channel_read_only_mode | OFF                                  |
| group_replication_auto_evict_timeout                      | 0                                  |
| group_replication_auto_increment_increment                | 7                                  |
| group_replication_autorejoin_tries                         | 288                                  |
| group_replication_bootstrap_group                         | OFF                                  |
| group_replication_broadcast_gtid_executed_period          | 1000                               |
| group_replication_clone_threshold                         | 9223372036854775807                |
| group_replication_communication_debug_options             | GCS_DEBUG_NONE                      |
| group_replication_communication_flp_timeout                | 5                                  |
| group_replication_communication_max_message_size          | 10485760                            |
| group_replication_communication_stack                      | MYSQL                               |
| group_replication_components_stop_timeout                   | 300                                  |
| group_replication_compression_threshold                   | 1000000                            |
| group_replication_consistency                               | EVENTUAL                            |
| group_replication_donor_threshold                         | 9223372036854775807                |
| group_replication_enforce_update_everywhere_checks          | OFF                                  |
| group_replication_exit_state_action                         | READ_ONLY                            |
| group_replication_flow_control_applier_threshold          | 25000                               |
| group_replication_flow_control_certifier_threshold          | 25000                               |
| group_replication_flow_control_hold_percent                | 10                                  |
| group_replication_flow_control_max_quota                   | 0                                  |
| group_replication_flow_control_max_wait_time                | 3600                               |
| group_replication_flow_control_member_quota_percent       | 0                                  |
| group_replication_flow_control_min_quota                   | 0                                  |
| group_replication_flow_control_min_recovery_quota          | 0                                  |
| group_replication_flow_control_mode                         | DISABLED                            |
| group_replication_flow_control_period                      | 1                                  |
| group_replication_flow_control_release_percent             | 50                                  |
| group_replication_flow_control_replay_lag_behind          | 600                                  |
| group_replication_force_members                            |                                     |
| group_replication_group_name                               | f2a0dd86-f159-11ee-ac63-08002751cf3a  |
| group_replication_group_seeds                               | 192.168.56.81:3306,192.168.56.82:3306 |
| group_replication_gtid_assignment_block_size                | 1000000                            |
| group_replication_ip_allowlist                            | AUTOMATIC                            |
| group_replication_ip_whitelist                            | AUTOMATIC                            |
| group_replication_local_address                            | 192.168.56.83:3306                   |
| group_replication_majority_after_mode                      | ON                                  |
| group_replication_member_expel_timeout                      | 5                                  |
| group_replication_member_weight                            | 50                                  |
| group_replication_message_cache_size                      | 1073741824                         |
| group_replication_paxos_single_leader                      | OFF                                  |
| group_replication_poll_spin_loops                         | 0                                  |
| group_replication_primary_election_mode                   | GTID_FIRST                         |
| group_replication_recovery_complete_at                      | TRANSACTIONS_APPLIED                |
| group_replication_recovery_compression_algorithms          | uncompressed                         |
| group_replication_recovery_get_public_key                   | ON                                  |
| group_replication_recovery_public_key_path                |                                     |
| group_replication_recovery_reconnect_interval             | 60                                  |
| group_replication_recovery_retry_count                      | 10                                  |
| group_replication_recovery_ssl_ca                         |                                     |
| group_replication_recovery_ssl_capath                      |                                     |
| group_replication_recovery_ssl_cert                         |                                     |
| group_replication_recovery_ssl_cipher                      |                                     |
| group_replication_recovery_ssl_crl                         |                                     |
| group_replication_recovery_ssl_crlpath                      |                                     |
| group_replication_recovery_ssl_key                         |                                     |
| group_replication_recovery_ssl_verify_server_cert          | OFF                                  |
| group_replication_recovery_tls_ciphersuites                |                                     |
| group_replication_recovery_tls_version                      | TLSv1.2,TLSv1.3                      |
| group_replication_recovery_use_ssl                         | ON                                  |
| group_replication_recovery_zstd_compression_level          | 3                                  |
| group_replication_request_time_threshold                   | 100                                  |
| group_replication_single_primary_fast_mode                | 0                                  |
| group_replication_single_primary_mode                      | ON                                  |
| group_replication_ssl_mode                                  | REQUIRED                            |
| group_replication_start_on_boot                            | ON                                  |
| group_replication_tls_source                               | MYSQL_MAIN                         |
| group_replication_transaction_size_limit                   | 150000000                            |
| group_replication_unreachable_majority_timeout             | 0                                  |
| group_replication_view_change_uuid                         | f2a0eb31-f159-11ee-ac63-08002751cf3a  |
| group_replication_xcom_cache_mode                         | 2                                  |
| group_replication_zone_id                                  | 0                                  |
| group_replication_zone_id_sync_mode                         | ON                                  |
+--------------------------------------------------------------+---------------------------------------+
77 rows in set (0.01 sec)

请问以上报错信息怎么解决？

全部回复(2)

yejr 2024-4-16 16:16:26

请提供3个节点上所有详细日志

回复采纳

景玉 2024-4-16 22:22:31

yejr 发表于 2024-4-16 16:16
请提供3个节点上所有详细日志

集群已经重构了，如果后面再遇到同样的问题，再来提问。

回复采纳

景玉

2 主题	0 博客	21 贡献

Rank: 1

积分: 27

互动与交流: 社区活动; 论坛问答; 加入社区; 贡献者指南

学习与应用: 文章博客; 文档资料; 视频资料; 常见问题

发现社区: 社区准则; 建议反馈; 社区之星

合作电话：010-64087828

社区邮箱：greatsql@greatdb.com

社区公众号

社区小助手

QQ群

友情链接: 万里数据库开源中国墨天轮开放原子开源基金会 OpenEuler

©2022 GreatSQL All Rights Reserved. ( 京ICP备06057874号 )

GMT+8, 2024-4-30 08:40 , Processed in 0.016625 second(s), 10 queries , Redis On.

快速回复 返回顶部 返回列表