MySQL MGR 8.0.27一主两从遇到其中一节点经常延迟
集群采用了MGR+ProxySQL的架构，正常上线一年中没有出现此类问题，最近ProxySQL节点经常看到有日志告警，2023-08-03 16:52:13 MySQL_Thread.cpp:3751:process_all_sessions(): Closing unhealthy client connection XXXXX
2023-08-03 16:52:19 MySQL_HostGroups_Manager.cpp:3406:group_replication_lag_action_set_server_status(): Shunning 'soft' server 1000:10.92.170.30:3306 with replication lag, count number: 3
2023-08-03 16:52:20 MySQL_Thread.cpp:3751:process_all_sessions(): Closing unhealthy client connection XXXXX
经排查所有的日志只显示10.92.170.30一个节点因为延迟被ProxySQL踢出代理服务，该业务主机采用的32C 64GB配置，业务量不大，主机CPU/内存/IO没有明显波动，请大佬帮忙给点建议，谢谢~
请补充几个信息
1. Primary和有延迟的Secondary节点上数据库尤其是MGR相关配置参数，执行show global variables查看
2. Primary和有延迟的Secondary节点上数据库负载信息，执行show processlist 和 show global status 查看
3. Primary和有延迟的Secondary节点上系统负载信息，执行vmstat -S m 1 300查看 yejr 发表于 2023-8-7 10:09
请补充几个信息
1. Primary和有延迟的Secondary节点上数据库尤其是MGR相关配置参数，执行show global varia ...
延迟Secondary节点信息
show global variables
activate_all_roles_on_login OFF
admin_address 127.0.0.1
admin_port 33062
admin_ssl_ca
admin_ssl_capath
admin_ssl_cert
admin_ssl_cipher
admin_ssl_crl
admin_ssl_crlpath
admin_ssl_key
admin_tls_ciphersuites
admin_tls_version TLSv1,TLSv1.1,TLSv1.2,TLSv1.3
authentication_policy *,,
auto_generate_certs ON
auto_increment_increment 1
auto_increment_offset 2
autocommit ON
automatic_sp_privileges ON
avoid_temporal_upgrade OFF
back_log 3000
basedir /usr/local/mysql/
big_tables OFF
bind_address *
binlog_cache_size 4194304
binlog_checksum CRC32
binlog_direct_non_transactional_updates OFF
binlog_encryption OFF
binlog_error_action ABORT_SERVER
binlog_expire_logs_seconds 604800
binlog_format ROW
binlog_group_commit_sync_delay 0
binlog_group_commit_sync_no_delay_count 0
binlog_gtid_simple_recovery ON
binlog_max_flush_queue_time 0
binlog_order_commits ON
binlog_rotate_encryption_master_key_at_startup OFF
binlog_row_event_max_size 8192
binlog_row_image FULL
binlog_row_metadata MINIMAL
binlog_row_value_options
binlog_rows_query_log_events OFF
binlog_stmt_cache_size 32768
binlog_transaction_compression OFF
binlog_transaction_compression_level_zstd 3
binlog_transaction_dependency_history_size 25000
binlog_transaction_dependency_tracking WRITESET
block_encryption_mode aes-128-ecb
bulk_insert_buffer_size 33554432
caching_sha2_password_auto_generate_rsa_keys ON
caching_sha2_password_digest_rounds 5000
caching_sha2_password_private_key_path private_key.pem
caching_sha2_password_public_key_path public_key.pem
character_set_client utf8mb4
character_set_connection utf8mb4
character_set_database utf8mb4
character_set_filesystem binary
character_set_results utf8mb4
character_set_server utf8mb4
character_set_system utf8mb3
character_sets_dir /usr/local/mysql/share/charsets/
check_proxy_users OFF
clone_autotune_concurrency ON
clone_block_ddl OFF
clone_buffer_size 4194304
clone_ddl_timeout 300
clone_donor_timeout_after_network_failure 5
clone_enable_compression OFF
clone_max_concurrency 16
clone_max_data_bandwidth 0
clone_max_network_bandwidth 0
clone_ssl_ca
clone_ssl_cert
clone_ssl_key
clone_valid_donor_list
collation_connection utf8mb4_general_ci
collation_database utf8mb4_general_ci
collation_server utf8mb4_general_ci
completion_type NO_CHAIN
concurrent_insert AUTO
connect_timeout 10
core_file OFF
create_admin_listener_thread OFF
cte_max_recursion_depth 1000
datadir /mysql/3306/data/
default_authentication_plugin mysql_native_password
default_collation_for_utf8mb4 utf8mb4_0900_ai_ci
default_password_lifetime 0
default_storage_engine InnoDB
default_table_encryption OFF
default_tmp_storage_engine InnoDB
default_week_format 0
delay_key_write ON
delayed_insert_limit 100
delayed_insert_timeout 300
delayed_queue_size 1000
disabled_storage_engines MyISAM,FEDERATED
disconnect_on_expired_password ON
div_precision_increment 4
end_markers_in_json OFF
enforce_gtid_consistency ON
eq_range_index_dive_limit 200
event_scheduler OFF
expire_logs_days 0
explicit_defaults_for_timestamp ON
flush OFF
flush_time 0
foreign_key_checks ON
ft_boolean_syntax + -><()~*:""&|
ft_max_word_len 84
ft_min_word_len 4
ft_query_expansion_limit 20
ft_stopword_file (built-in)
general_log OFF
general_log_file /mysql/3306/data/p-mysql-3.log
generated_random_password_length 20
group_concat_max_len 1024
group_replication_advertise_recovery_endpoints DEFAULT
group_replication_allow_local_lower_version_join OFF
group_replication_auto_increment_increment 7
group_replication_autorejoin_tries 3
group_replication_bootstrap_group OFF
group_replication_clone_threshold 9223372036854775807
group_replication_communication_debug_options GCS_DEBUG_NONE
group_replication_communication_max_message_size 10485760
group_replication_communication_stack XCOM
group_replication_components_stop_timeout 300
group_replication_compression_threshold 1000000
group_replication_consistency BEFORE
group_replication_enforce_update_everywhere_checks OFF
group_replication_exit_state_action READ_ONLY
group_replication_flow_control_applier_threshold 25000
group_replication_flow_control_certifier_threshold 25000
group_replication_flow_control_hold_percent 10
group_replication_flow_control_max_quota 0
group_replication_flow_control_member_quota_percent 0
group_replication_flow_control_min_quota 0
group_replication_flow_control_min_recovery_quota 0
group_replication_flow_control_mode QUOTA
group_replication_flow_control_period 1
group_replication_flow_control_release_percent 50
group_replication_force_members
group_replication_group_name 205fa831-9e8d-11ec-aa8a-005056aad771
group_replication_group_seeds 10.92.170.28:33061,10.92.170.29:33061
group_replication_gtid_assignment_block_size 1000000
group_replication_ip_allowlist AUTOMATIC
group_replication_ip_whitelist AUTOMATIC
group_replication_local_address 10.92.170.30:33061
group_replication_member_expel_timeout 5
group_replication_member_weight 50
group_replication_message_cache_size 1073741824
group_replication_paxos_single_leader OFF
group_replication_poll_spin_loops 0
group_replication_recovery_complete_at TRANSACTIONS_APPLIED
group_replication_recovery_compression_algorithms uncompressed
group_replication_recovery_get_public_key OFF
group_replication_recovery_public_key_path
group_replication_recovery_reconnect_interval 60
group_replication_recovery_retry_count 10
group_replication_recovery_ssl_ca
group_replication_recovery_ssl_capath
group_replication_recovery_ssl_cert
group_replication_recovery_ssl_cipher
group_replication_recovery_ssl_crl
group_replication_recovery_ssl_crlpath
group_replication_recovery_ssl_key
group_replication_recovery_ssl_verify_server_cert OFF
group_replication_recovery_tls_ciphersuites
group_replication_recovery_tls_version TLSv1,TLSv1.1,TLSv1.2,TLSv1.3
group_replication_recovery_use_ssl ON
group_replication_recovery_zstd_compression_level 3
group_replication_single_primary_mode ON
group_replication_ssl_mode REQUIRED
group_replication_start_on_boot ON
group_replication_tls_source MYSQL_MAIN
group_replication_transaction_size_limit 150000000
group_replication_unreachable_majority_timeout 0
group_replication_view_change_uuid 205faf7b-9e8d-11ec-aa8a-005056aad771
gtid_executed 205fa831-9e8d-11ec-aa8a-005056aad771:1-881173413,
205faf7b-9e8d-11ec-aa8a-005056aad771:1-22
gtid_executed_compression_period 0
gtid_mode ON
gtid_owned
gtid_purged 205fa831-9e8d-11ec-aa8a-005056aad771:1-855878461,
205faf7b-9e8d-11ec-aa8a-005056aad771:1-22
have_compress YES
have_dynamic_loading YES
have_geometry YES
have_openssl YES
have_profiling YES
have_query_cache NO
have_rtree_keys YES
have_ssl YES
have_statement_timeout YES
have_symlink DISABLED
histogram_generation_max_mem_size 20000000
host_cache_size 753
hostname p-mysql-3
information_schema_stats_expiry 86400
init_connect
init_file
init_replica
init_slave
innodb_adaptive_flushing ON
innodb_adaptive_flushing_lwm 10
innodb_adaptive_hash_index ON
innodb_adaptive_hash_index_parts 8
innodb_adaptive_max_sleep_delay 150000
innodb_api_bk_commit_interval 5
innodb_api_disable_rowlock OFF
innodb_api_enable_binlog OFF
innodb_api_enable_mdl OFF
innodb_api_trx_level 0
innodb_autoextend_increment 64
innodb_autoinc_lock_mode 2
innodb_buffer_pool_chunk_size 134217728
innodb_buffer_pool_dump_at_shutdown ON
innodb_buffer_pool_dump_now OFF
innodb_buffer_pool_dump_pct 25
innodb_buffer_pool_filename ib_buffer_pool
innodb_buffer_pool_in_core_file ON
innodb_buffer_pool_instances 1
innodb_buffer_pool_load_abort OFF
innodb_buffer_pool_load_at_startup ON
innodb_buffer_pool_load_now OFF
innodb_buffer_pool_size 13958643712
innodb_change_buffer_max_size 25
innodb_change_buffering all
innodb_checksum_algorithm crc32
innodb_cmp_per_index_enabled OFF
innodb_commit_concurrency 0
innodb_compression_failure_threshold_pct 5
innodb_compression_level 6
innodb_compression_pad_pct_max 50
innodb_concurrency_tickets 5000
innodb_data_file_path ibdata1:100M:autoextend
innodb_data_home_dir
innodb_ddl_buffer_size 1048576
innodb_ddl_threads 4
innodb_deadlock_detect ON
innodb_dedicated_server OFF
innodb_default_row_format dynamic
innodb_directories
innodb_disable_sort_file_cache OFF
innodb_doublewrite ON
innodb_doublewrite_batch_size 0
innodb_doublewrite_dir
innodb_doublewrite_files 2
innodb_doublewrite_pages 4
innodb_extend_and_initialize ON
innodb_fast_shutdown 1
innodb_file_per_table ON
innodb_fill_factor 100
innodb_flush_log_at_timeout 1
innodb_flush_log_at_trx_commit 1
innodb_flush_method O_DIRECT
innodb_flush_neighbors 0
innodb_flush_sync ON
innodb_flushing_avg_loops 30
innodb_force_load_corrupted OFF
innodb_force_recovery 0
innodb_fsync_threshold 0
innodb_ft_aux_table
innodb_ft_cache_size 8000000
innodb_ft_enable_diag_print OFF
innodb_ft_enable_stopword ON
innodb_ft_max_token_size 84
innodb_ft_min_token_size 3
innodb_ft_num_word_optimize 2000
innodb_ft_result_cache_limit 2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree 2
innodb_ft_total_cache_size 640000000
innodb_ft_user_stopword_table
innodb_idle_flush_pct 100
innodb_io_capacity 4000
innodb_io_capacity_max 8000
innodb_lock_wait_timeout 50
innodb_log_buffer_size 33554432
innodb_log_checksums ON
innodb_log_compressed_pages ON
innodb_log_file_size 1048576000
innodb_log_files_in_group 3
innodb_log_group_home_dir ./
innodb_log_spin_cpu_abs_lwm 80
innodb_log_spin_cpu_pct_hwm 50
innodb_log_wait_for_flush_spin_hwm 400
innodb_log_write_ahead_size 8192
innodb_log_writer_threads ON
innodb_lru_scan_depth 1024
innodb_max_dirty_pages_pct 50.000000
innodb_max_dirty_pages_pct_lwm 10.000000
innodb_max_purge_lag 0
innodb_max_purge_lag_delay 0
innodb_max_undo_log_size 1073741824
innodb_monitor_disable
innodb_monitor_enable module_buffer_page
innodb_monitor_reset
innodb_monitor_reset_all
innodb_numa_interleave OFF
innodb_old_blocks_pct 37
innodb_old_blocks_time 1000
innodb_online_alter_log_max_size 134217728
innodb_open_files 2048
innodb_optimize_fulltext_only OFF
innodb_page_cleaners 1
innodb_page_size 16384
innodb_parallel_read_threads 4
innodb_print_all_deadlocks OFF
innodb_print_ddl_logs OFF
innodb_purge_batch_size 300
innodb_purge_rseg_truncate_frequency 128
innodb_purge_threads 4
innodb_random_read_ahead OFF
innodb_read_ahead_threshold 56
innodb_read_io_threads 4
innodb_read_only OFF
innodb_redo_log_archive_dirs
innodb_redo_log_encrypt OFF
innodb_replication_delay 0
innodb_rollback_on_timeout ON
innodb_rollback_segments 128
innodb_segment_reserve_factor 12.500000
innodb_sort_buffer_size 1048576
innodb_spin_wait_delay 6
innodb_spin_wait_pause_multiplier 50
innodb_stats_auto_recalc ON
innodb_stats_include_delete_marked OFF
innodb_stats_method nulls_equal
innodb_stats_on_metadata OFF
innodb_stats_persistent ON
innodb_stats_persistent_sample_pages 20
innodb_stats_transient_sample_pages 8
innodb_status_output OFF
innodb_status_output_locks OFF
innodb_strict_mode ON
innodb_sync_array_size 1
innodb_sync_spin_loops 30
innodb_table_locks ON
innodb_temp_data_file_path ibtmp1:12M:autoextend
innodb_temp_tablespaces_dir ./#innodb_temp/
innodb_thread_concurrency 0
innodb_thread_sleep_delay 10000
innodb_tmpdir
innodb_undo_directory ./
innodb_undo_log_encrypt OFF
innodb_undo_log_truncate ON
innodb_undo_tablespaces 2
innodb_use_fdatasync OFF
innodb_use_native_aio ON
innodb_validate_tablespace_paths ON
innodb_version 8.0.27
innodb_write_io_threads 4
interactive_timeout 3600
internal_tmp_mem_storage_engine TempTable
join_buffer_size 4194304
keep_files_on_create OFF
key_buffer_size 8388608
key_cache_age_threshold 300
key_cache_block_size 1024
key_cache_division_limit 100
keyring_operations ON
large_files_support ON
large_page_size 0
large_pages OFF
lc_messages en_US
lc_messages_dir /usr/local/mysql/share/
lc_time_names en_US
license GPL
local_infile OFF
lock_wait_timeout 3600
locked_in_memory OFF
log_bin ON
log_bin_basename /mysql/3306/log/mysql-bin
log_bin_index /mysql/3306/log/mysql-bin.index
log_bin_trust_function_creators ON
log_bin_use_v1_row_events OFF
log_error /mysql/3306/log/mysql-error.log
log_error_services log_filter_internal; log_sink_internal
log_error_suppression_list
log_error_verbosity 3
log_output FILE
log_queries_not_using_indexes ON
log_raw OFF
log_replica_updates ON
log_slave_updates ON
log_slow_admin_statements OFF
log_slow_extra ON
log_slow_replica_statements ON
log_slow_slave_statements ON
log_statements_unsafe_for_binlog ON
log_throttle_queries_not_using_indexes 10
log_timestamps SYSTEM
long_query_time 1.000000
low_priority_updates OFF
lower_case_file_system OFF
lower_case_table_names 1
mandatory_roles
master_info_repository TABLE
master_verify_checksum OFF
max_allowed_packet 67108864
max_binlog_cache_size 18446744073709547520
max_binlog_size 134217728
max_binlog_stmt_cache_size 18446744073709547520
max_connect_errors 100000
max_connections 3000
max_delayed_threads 20
max_digest_length 1024
max_error_count 1024
max_execution_time 0
max_heap_table_size 100663296
max_insert_delayed_threads 20
max_join_size 18446744073709551615
max_length_for_sort_data 4096
max_points_in_geometry 65536
max_prepared_stmt_count 16382
max_relay_log_size 134217728
max_seeks_for_key 18446744073709551615
max_sort_length 1024
max_sp_recursion_depth 0
max_user_connections 0
max_write_lock_count 18446744073709551615
min_examined_row_limit 0
myisam_data_pointer_size 6
myisam_max_sort_file_size 10737418240
myisam_mmap_size 18446744073709551615
myisam_recover_options OFF
myisam_repair_threads 1
myisam_sort_buffer_size 134217728
myisam_stats_method nulls_unequal
myisam_use_mmap OFF
mysql_native_password_proxy_users OFF
mysqlx_bind_address *
mysqlx_compression_algorithms DEFLATE_STREAM,LZ4_MESSAGE,ZSTD_STREAM
mysqlx_connect_timeout 30
mysqlx_deflate_default_compression_level 3
mysqlx_deflate_max_client_compression_level 5
mysqlx_document_id_unique_prefix 0
mysqlx_enable_hello_notice ON
mysqlx_idle_worker_thread_timeout 60
mysqlx_interactive_timeout 28800
mysqlx_lz4_default_compression_level 2
mysqlx_lz4_max_client_compression_level 8
mysqlx_max_allowed_packet 67108864
mysqlx_max_connections 100
mysqlx_min_worker_threads 2
mysqlx_port 33060
mysqlx_port_open_timeout 0
mysqlx_read_timeout 30
mysqlx_socket /tmp/mysqlx.sock
mysqlx_ssl_ca
mysqlx_ssl_capath
mysqlx_ssl_cert
mysqlx_ssl_cipher
mysqlx_ssl_crl
mysqlx_ssl_crlpath
mysqlx_ssl_key
mysqlx_wait_timeout 28800
mysqlx_write_timeout 60
mysqlx_zstd_default_compression_level 3
mysqlx_zstd_max_client_compression_level 11
net_buffer_length 16384
net_read_timeout 30
net_retry_count 10
net_write_timeout 60
new OFF
ngram_token_size 2
offline_mode OFF
old OFF
old_alter_table OFF
open_files_limit 65535
optimizer_prune_level 1
optimizer_search_depth 62
optimizer_switch index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on,use_invisible_indexes=off,skip_scan=on,hash_join=on,subquery_to_derived=off,prefer_ordering_index=on,hypergraph_optimizer=off,derived_condition_pushdown=on
optimizer_trace enabled=off,one_line=off
optimizer_trace_features greedy_search=on,range_optimizer=on,dynamic_range=on,repeated_subselect=on
optimizer_trace_limit 1
optimizer_trace_max_mem_size 1048576
optimizer_trace_offset -1
parser_max_mem_size 18446744073709551615
partial_revokes OFF
password_history 0
password_require_current OFF
password_reuse_interval 0
performance_schema ON
performance_schema_accounts_size -1
performance_schema_digests_size 10000
performance_schema_error_size 5018
performance_schema_events_stages_history_long_size 10000
performance_schema_events_stages_history_size 10
performance_schema_events_statements_history_long_size 10000
performance_schema_events_statements_history_size 10
performance_schema_events_transactions_history_long_size 10000
performance_schema_events_transactions_history_size 10
performance_schema_events_waits_history_long_size 10000
performance_schema_events_waits_history_size 10
performance_schema_hosts_size -1
performance_schema_max_cond_classes 150
performance_schema_max_cond_instances -1
performance_schema_max_digest_length 1024
performance_schema_max_digest_sample_age 60
performance_schema_max_file_classes 80
performance_schema_max_file_handles 32768
performance_schema_max_file_instances -1
performance_schema_max_index_stat -1
performance_schema_max_memory_classes 450
performance_schema_max_metadata_locks -1
performance_schema_max_mutex_classes 350
performance_schema_max_mutex_instances -1
performance_schema_max_prepared_statements_instances -1
performance_schema_max_program_instances -1
performance_schema_max_rwlock_classes 60
performance_schema_max_rwlock_instances -1
performance_schema_max_socket_classes 10
performance_schema_max_socket_instances -1
performance_schema_max_sql_text_length 1024
performance_schema_max_stage_classes 175
performance_schema_max_statement_classes 219
performance_schema_max_statement_stack 10
performance_schema_max_table_handles -1
performance_schema_max_table_instances -1
performance_schema_max_table_lock_stat -1
performance_schema_max_thread_classes 100
performance_schema_max_thread_instances -1
performance_schema_session_connect_attrs_size 512
performance_schema_setup_actors_size -1
performance_schema_setup_objects_size -1
performance_schema_show_processlist OFF
performance_schema_users_size -1
persist_only_admin_x509_subject
persisted_globals_load ON
pid_file /mysql/3306/data/mysql.pid
plugin_dir /usr/local/mysql/lib/plugin/
port 3306
preload_buffer_size 32768
print_identified_with_as_hex OFF
profiling OFF
profiling_history_size 15
protocol_compression_algorithms zlib,zstd,uncompressed
protocol_version 10
query_alloc_block_size 8192
query_prealloc_size 8192
range_alloc_block_size 4096
range_optimizer_max_mem_size 8388608
rbr_exec_mode STRICT
read_buffer_size 8388608
read_only ON
read_rnd_buffer_size 16777216
regexp_stack_limit 8000000
regexp_time_limit 32
relay_log /mysql/3306/log/relay-bin
relay_log_basename /mysql/3306/log/relay-bin
relay_log_index /mysql/3306/data/relay-bin.index
relay_log_info_file relay-log.info
relay_log_info_repository TABLE
relay_log_purge ON
relay_log_recovery ON
relay_log_space_limit 0
replica_allow_batching OFF
replica_checkpoint_group 512
replica_checkpoint_period 2
replica_compressed_protocol OFF
replica_exec_mode STRICT
replica_load_tmpdir /mysql/3306/tmp
replica_max_allowed_packet 1073741824
replica_net_timeout 60
replica_parallel_type LOGICAL_CLOCK
replica_parallel_workers 4
replica_pending_jobs_size_max 134217728
replica_preserve_commit_order ON
replica_skip_errors OFF
replica_sql_verify_checksum ON
replica_transaction_retries 10
replica_type_conversions
replication_optimize_for_static_plugin_config OFF
replication_sender_observe_commit_only OFF
report_host 10.92.170.30
report_password
report_port 3306
report_user
require_secure_transport OFF
rpl_read_size 8192
rpl_semi_sync_master_enabled ON
rpl_semi_sync_master_timeout 1000
rpl_semi_sync_master_trace_level 32
rpl_semi_sync_master_wait_for_slave_count 1
rpl_semi_sync_master_wait_no_slave ON
rpl_semi_sync_master_wait_point AFTER_SYNC
rpl_semi_sync_slave_enabled ON
rpl_semi_sync_slave_trace_level 32
rpl_stop_replica_timeout 31536000
rpl_stop_slave_timeout 31536000
schema_definition_cache 256
secondary_engine_cost_threshold 100000.000000
secure_file_priv /tmp/
select_into_buffer_size 131072
select_into_disk_sync OFF
select_into_disk_sync_delay 0
server_id 303306
server_id_bits 32
server_uuid 6aa8e2c8-66e8-11ec-9edd-005056a77730
session_track_gtids OFF
session_track_schema ON
session_track_state_change OFF
session_track_system_variables time_zone,autocommit,character_set_client,character_set_results,character_set_connection
session_track_transaction_info OFF
sha256_password_auto_generate_rsa_keys ON
sha256_password_private_key_path private_key.pem
sha256_password_proxy_users OFF
sha256_password_public_key_path public_key.pem
show_create_table_verbosity OFF
show_old_temporals OFF
skip_external_locking ON
skip_name_resolve ON
skip_networking OFF
skip_replica_start OFF
skip_show_database OFF
skip_slave_start OFF
slave_allow_batching OFF
slave_checkpoint_group 512
slave_checkpoint_period 2
slave_compressed_protocol OFF
slave_exec_mode STRICT
slave_load_tmpdir /mysql/3306/tmp
slave_max_allowed_packet 1073741824
slave_net_timeout 60
slave_parallel_type LOGICAL_CLOCK
slave_parallel_workers 4
slave_pending_jobs_size_max 134217728
slave_preserve_commit_order ON
slave_rows_search_algorithms INDEX_SCAN,HASH_SCAN
slave_skip_errors OFF
slave_sql_verify_checksum ON
slave_transaction_retries 10
slave_type_conversions
slow_launch_time 2
slow_query_log ON
slow_query_log_file /mysql/3306/log/mysql-slow.log
socket /mysql/3306/tmp/mysql.sock
sort_buffer_size 4194304
source_verify_checksum OFF
sql_auto_is_null OFF
sql_big_selects ON
sql_buffer_result OFF
sql_log_off OFF
sql_mode NO_ENGINE_SUBSTITUTION
sql_notes ON
sql_quote_show_create ON
sql_replica_skip_counter 0
sql_require_primary_key ON
sql_safe_updates OFF
sql_select_limit 18446744073709551615
sql_slave_skip_counter 0
sql_warnings OFF
ssl_ca ca.pem
ssl_capath
ssl_cert server-cert.pem
ssl_cipher
ssl_crl
ssl_crlpath
ssl_fips_mode OFF
ssl_key server-key.pem
stored_program_cache 256
stored_program_definition_cache 256
super_read_only ON
sync_binlog 1
sync_master_info 10000
sync_relay_log 10000
sync_relay_log_info 10000
sync_source_info 10000
system_time_zone CST
table_definition_cache 2048
table_encryption_privilege_check OFF
table_open_cache 2048
table_open_cache_instances 16
tablespace_definition_cache 256
temptable_max_mmap 1073741824
temptable_max_ram 1073741824
temptable_use_mmap ON
terminology_use_previous NONE
thread_cache_size 200
thread_handling one-thread-per-connection
thread_stack 1048576
time_zone +08:00
tls_ciphersuites
tls_version TLSv1,TLSv1.1,TLSv1.2,TLSv1.3
tmp_table_size 100663296
tmpdir /mysql/3306/tmp
transaction_alloc_block_size 8192
transaction_isolation READ-COMMITTED
transaction_prealloc_size 4096
transaction_read_only OFF
transaction_write_set_extraction XXHASH64
unique_checks ON
updatable_views_with_limit YES
version 8.0.27
version_comment MySQL Community Server - GPL
version_compile_machine x86_64
version_compile_os Linux
version_compile_zlib 1.2.11
wait_timeout 3600
windowing_use_high_precision ON shawn 发表于 2023-8-7 10:21
延迟Secondary节点信息
show global variables
show processlist
10 system user Connect 21428318 waiting for handler commit Group replication applier module
12 system user Query 0 Replica has read all relay log; waiting for more updates
13 system user Query 0 Waiting for an event from Coordinator
14 system user Query 0 Waiting for an event from Coordinator
15 system user Query 0 Waiting for an event from Coordinator
16 system user Query 3 Waiting for an event from Coordinator
2832994 repl_user 10.76.12.34:47034 Binlog Dump GTID 578667 Source has sent all binlog to replica; waiting for more updates
2934372 rw_platform 10.92.170.49:45268 platform Sleep 1172
2938600 rw_platform 10.92.170.48:49880 file Sleep 3
2941110 rw_platform 10.92.170.49:54304 platform Sleep 27
2941279 rw_platform 10.92.170.47:33482 platform Sleep 24
2941512 rw_platform 10.92.170.48:53782 platform Sleep 961
2941515 rw_platform 10.92.170.47:33732 file Sleep 3
2941517 rw_platform 10.92.170.49:54738 platform Sleep 0
2941518 rw_platform 10.92.170.47:33734 file Sleep 2
2941519 rw_platform 10.92.170.48:53788 platform Sleep 5
2941522 rw_platform 10.92.170.49:54740 platform Sleep 0
2941523 rw_platform 10.92.170.47:33738 file Sleep 3
2941526 rw_platform 10.92.170.48:53794 admin Sleep 4
2941527 rw_platform 10.92.170.47:33742 platform Sleep 1
2941529 rw_platform 10.92.170.49:54742 admin Sleep 3
2941530 rw_platform 10.92.170.48:53798 platform Sleep 21
2941531 rw_platform 10.92.170.49:54744 scheduler Sleep 3
2941532 rw_platform 10.92.170.47:33744 scheduler Sleep 1
2941534 rw_platform 10.92.170.49:54746 platform Sleep 0
2941535 rw_platform 10.92.170.47:33746 scheduler Sleep 3
2941536 rw_platform 10.92.170.48:53802 platform Sleep 4
2941537 rw_platform 10.92.170.49:54748 scheduler Sleep 1
2941538 rw_platform 10.92.170.47:33748 platform Sleep 15
2941539 rw_platform 10.92.170.48:53804 file Sleep 43
2941542 rw_platform 10.92.170.49:54752 platform Sleep 0
2941545 rw_platform 10.92.170.49:54756 scheduler Sleep 0
2941546 rw_platform 10.92.170.47:33754 file Sleep 3
2941551 rw_platform 10.92.170.48:53816 platform Sleep 7
2941552 rw_platform 10.92.170.49:54762 platform Sleep 3
2941553 rw_platform 10.92.170.47:33756 scheduler Sleep 1
2941556 rw_platform 10.92.170.47:33758 platform Sleep 0
2941559 rw_platform 10.92.170.49:54768 scheduler Sleep 1
2941560 rw_platform 10.92.170.47:33762 file Sleep 3
2941561 rw_platform 10.92.170.49:54770 file Sleep 43
2941562 rw_platform 10.92.170.47:33764 file Sleep 3
2941564 rw_platform 10.92.170.49:54772 platform Sleep 9
2941565 rw_platform 10.92.170.47:33766 platform Sleep 2
2941566 rw_platform 10.92.170.47:33768 file Sleep 3
2941567 rw_platform 10.92.170.47:33770 scheduler Sleep 1
2941568 rw_platform 10.92.170.48:53824 file Sleep 3
2941570 rw_platform 10.92.170.47:33772 platform Sleep 29
2941572 rw_platform 10.92.170.47:33774 scheduler Sleep 1
2941574 rw_platform 10.92.170.48:53826 platform Sleep 0
2941576 rw_platform 10.92.170.47:33776 import Sleep 3
2941577 rw_platform 10.92.170.49:54782 file Sleep 3
2941579 rw_platform 10.92.170.49:54784 scheduler Sleep 1
2941580 rw_platform 10.92.170.49:54786 platform Sleep 13
2941581 rw_platform 10.92.170.47:33778 file Sleep 3
2941582 rw_platform 10.92.170.47:33780 platform Sleep 10
2941583 rw_platform 10.92.170.48:53830 file Sleep 3
2941584 rw_platform 10.92.170.49:54788 file Sleep 1
2941586 rw_platform 10.92.170.49:54790 platform Sleep 0
2941588 rw_platform 10.92.170.49:54792 file Sleep 3
2941591 rw_platform 10.92.170.47:33784 file Sleep 43
2941593 rw_platform 10.92.170.49:54796 platform Sleep 17
2941594 rw_platform 10.92.170.47:33786 platform Sleep 3
2941597 rw_platform 10.92.170.49:54798 platform Sleep 0
2941598 rw_platform 10.92.170.49:54800 platform Sleep 0
2941599 rw_platform 10.92.170.47:33788 platform Sleep 5
2941600 rw_platform 10.92.170.48:53842 platform Sleep 24
2941602 rw_platform 10.92.170.48:53846 admin Sleep 3
2941604 rw_platform 10.92.170.47:33794 platform Sleep 0
2941607 rw_platform 10.92.170.49:54808 scheduler Sleep 3
2941608 rw_platform 10.92.170.49:54810 import Sleep 1
2941616 rw_seata 10.92.170.47:33806 seata Sleep 1
2941617 rw_seata 10.92.170.49:54822 seata Sleep 1
2941621 rw_seata 10.92.170.49:54830 seata Sleep 0
2941653 rw_seata 10.92.170.47:33840 seata Sleep 3
2941655 rw_platform 10.92.170.49:54862 file Sleep 43
2941656 rw_platform 10.92.170.49:54864 scheduler Sleep 1
2941659 rw_platform 10.92.170.48:53904 admin Sleep 318
2941661 rw_platform 10.92.170.49:54866 import Sleep 1
2941662 rw_platform 10.92.170.48:53908 platform Sleep 3
2941663 rw_platform 10.92.170.49:54868 platform Sleep 0
2941668 rw_platform 10.92.170.49:54878 platform Sleep 14
2941670 rw_platform 10.92.170.49:54882 platform Sleep 0
2941671 rw_platform 10.92.170.49:54884 platform Sleep 6
2941672 zabbix 10.92.170.47:33858 information_schema Sleep 1
2941673 zabbix 10.92.170.49:54886 information_schema Sleep 1
2941674 rw_platform 10.92.170.49:54890 platform Sleep 18
2941675 rw_platform 10.92.170.49:54892 platform Sleep 0
2941676 rw_platform 10.92.170.49:54894 import Sleep 1
2941677 rw_platform 10.92.170.48:53918 platform Sleep 3
2941678 rw_platform 10.92.170.49:54896 platform Sleep 8
2941680 rw_platform 10.92.170.49:54898 platform Sleep 1
2941682 rw_platform 10.92.170.48:53924 scheduler Sleep 3
2941683 rw_platform 10.92.170.49:54900 file Sleep 43
2941684 rw_platform 10.92.170.47:33860 platform Sleep 21
2941685 rw_platform 10.92.170.47:33862 platform Sleep 19
2941686 rw_platform 10.92.170.47:33864 file Sleep 1
2941688 rw_platform 10.92.170.49:54902 platform Sleep 0
2941689 rw_platform 10.92.170.47:33866 scheduler Sleep 1
2941690 rw_platform 10.92.170.49:54904 platform Sleep 0
2941691 rw_platform 10.92.170.47:33868 scheduler Sleep 1
2941692 rw_platform 10.92.170.48:53928 platform Sleep 5
2941693 rw_platform 10.92.170.47:33870 file Sleep 43
2941694 rw_platform 10.92.170.49:54906 file Sleep 1
2941695 rw_platform 10.92.170.47:33872 scheduler Sleep 1
2941697 rw_platform 10.92.170.49:54908 platform Sleep 1
2941698 rw_platform 10.92.170.47:33874 platform Sleep 1
2941699 rw_platform 10.92.170.49:54910 platform Sleep 1
2941700 rw_platform 10.92.170.47:33876 platform Sleep 25
2941702 rw_platform 10.92.170.48:53932 admin Sleep 4
2941703 rw_platform 10.92.170.48:53934 import Sleep 4
2941704 rw_platform 10.92.170.48:53936 scheduler Sleep 1
2941705 rw_platform 10.92.170.47:33880 file Sleep 3
2941706 rw_platform 10.92.170.47:33882 import Sleep 3
2941707 rw_platform 10.92.170.49:54912 file Sleep 0
2941708 rw_platform 10.92.170.47:33884 platform Sleep 1236
2941709 rw_platform 10.92.170.47:33886 import Sleep 1759
2941710 rw_platform 10.92.170.47:33888 file Sleep 3
2941711 rw_platform 10.92.170.49:54914 scheduler Sleep 3
2941712 rw_platform 10.92.170.47:33890 platform Sleep 3
2941713 rw_platform 10.92.170.49:54916 platform Sleep 11
2941714 rw_platform 10.92.170.47:33892 platform Sleep 3
2941715 rw_platform 10.92.170.47:33894 scheduler Sleep 1
2941716 rw_platform 10.92.170.47:33896 platform Sleep 1
2941724 rw_platform 10.92.170.47:33910 platform Sleep 8
2941725 rw_platform 10.92.170.47:33912 file Sleep 1
2941728 rw_platform 10.92.170.47:33916 admin Sleep 3
2941729 rw_platform 10.92.170.47:33918 platform Sleep 3
2941733 rw_platform 10.92.170.48:53962 platform Sleep 8
2941735 rw_platform 10.92.170.49:54934 platform Sleep 7
2941736 rw_platform 10.92.170.47:33922 platform Sleep 7
2941737 rw_platform 10.92.170.49:54938 platform Sleep 21
2941739 rw_platform 10.92.170.48:53968 platform Sleep 27
2941741 rw_platform 10.92.170.49:54940 platform Sleep 16
2941746 rw_platform 10.92.170.49:54950 platform Sleep 7
2941747 rw_platform 10.92.170.49:54952 platform Sleep 0
2941748 rw_platform 10.92.170.49:54954 scheduler Sleep 1
2941750 rw_platform 10.92.170.47:33934 platform Sleep 12
2941751 rw_platform 10.92.170.47:33936 platform Sleep 3
2941752 rw_platform 10.92.170.47:33940 platform Sleep 2
2941753 rw_platform 10.92.170.47:33942 platform Sleep 27
2941754 rw_platform 10.92.170.47:33944 platform Sleep 30
2941755 rw_platform 10.92.170.47:33946 scheduler Sleep 3
2941756 rw_platform 10.92.170.47:33948 scheduler Sleep 1
2941757 rw_platform 10.92.170.47:33950 platform Sleep 0
2941758 rw_seata 10.92.170.48:53984 seata Sleep 4
2941789 rw_platform 10.92.170.48:54010 admin Sleep 4
2941793 rw_platform 10.92.170.48:54022 platform Sleep 0
2941794 rw_platform 10.92.170.48:54024 file Sleep 3
2941795 rw_platform 10.92.170.48:54026 file Sleep 43
2941796 rw_platform 10.92.170.48:54028 platform Sleep 2
2941797 rw_platform 10.92.170.48:54030 scheduler Sleep 3
2941798 rw_platform 10.92.170.48:54032 platform Sleep 0
2941800 rw_platform 10.92.170.48:54036 platform Sleep 0
2941807 rw_platform 10.92.170.48:54050 file Sleep 43
2941809 rw_platform 10.92.170.49:54990 file Sleep 2
2941810 rw_platform 10.92.170.49:54992 platform Sleep 8
2941811 rw_nacos 10.92.170.48:54054 nacos Sleep 4
2941812 rw_platform 10.92.170.49:54994 file Sleep 1
2941821 rw_seata 10.92.170.47:33996 seata Sleep 3
2941822 rw_seata 10.92.170.47:33998 seata Sleep 1
2941823 rw_seata 10.92.170.47:34000 seata Sleep 3
2941833 rw_platform 10.92.170.48:54088 scheduler Sleep 3
2941834 rw_platform 10.92.170.48:54090 platform Sleep 1
2941835 rw_platform 10.92.170.48:54092 import Sleep 4
2941836 rw_platform 10.92.170.48:54096 scheduler Sleep 3
2941838 rw_platform 10.92.170.48:54102 platform Sleep 3
2941840 rw_platform 10.92.170.48:54106 platform Sleep 9
2941841 rw_platform 10.92.170.48:54108 platform Sleep 4
2941842 rw_platform 10.92.170.48:54110 platform Sleep 3
2941843 rw_platform 10.92.170.48:54112 scheduler Sleep 3
2941844 rw_seata 10.92.170.48:54114 seata Sleep 4
2941847 rw_platform 10.92.170.48:54116 scheduler Sleep 4
2941849 rw_platform 10.92.170.48:54118 platform Sleep 3
2941850 rw_platform 10.92.170.48:54120 platform Sleep 6
2941851 rw_platform 10.92.170.48:54122 platform Sleep 4
2941852 rw_platform 10.92.170.48:54124 platform Sleep 29
2941855 rw_platform 10.92.170.48:54128 platform Sleep 1
2941856 rw_platform 10.92.170.48:54130 platform Sleep 5
2941857 rw_platform 10.92.170.48:54132 import Sleep 4
2941889 rw_seata 10.92.170.49:55074 seata Sleep 47
2941890 rw_platform 10.92.170.48:54166 platform Sleep 21
2941891 rw_platform 10.92.170.48:54168 file Sleep 43
2941892 rw_nacos 10.92.170.49:55076 nacos Sleep 5
2941902 rw_nacos 10.92.170.47:34078 nacos Sleep 6
2941906 rw_seata 10.92.170.48:54196 seata Sleep 4
2941912 rw_seata 10.92.170.48:54200 seata Sleep 1
2941914 rw_platform 10.92.170.48:54206 platform Sleep 1
2941915 rw_platform 10.92.170.48:54208 scheduler Sleep 3
2941924 rw_seata 10.92.170.48:54226 seata Sleep 4
2941951 rw_platform 10.92.170.47:34128 platform Sleep 7
2941956 rw_nacos 10.92.170.49:55152 nacos Sleep 1
2941958 rw_platform 10.92.170.47:34140 scheduler Sleep 3
2941959 rw_platform 10.92.170.47:34142 file Sleep 0
2941960 rw_platform 10.92.170.48:54254 platform Sleep 22
2941961 rw_platform 10.92.170.48:54256 admin Sleep 1289
2942141 rw_platform 10.92.170.49:55414 platform Sleep 37
2942142 rw_platform 10.92.170.49:55416 file Sleep 2
2942144 rw_platform 10.92.170.49:55420 scheduler Sleep 1
2942145 rw_platform 10.92.170.49:55422 platform Sleep 1
2942146 rw_platform 10.92.170.49:55424 scheduler Sleep 1
2942147 rw_platform 10.92.170.49:55426 import Sleep 1
2942150 rw_platform 10.92.170.49:55430 scheduler Sleep 1
2942195 rw_nacos 10.92.170.47:34462 nacos Sleep 1
2942197 rw_seata 10.92.170.49:55490 seata Sleep 0
2942206 rw_platform 10.92.170.49:55506 platform Sleep 1
2942208 rw_platform 10.92.170.49:55508 platform Sleep 24
2942209 rw_platform 10.92.170.49:55510 file Sleep 1
2942210 rw_platform 10.92.170.49:55514 platform Sleep 3
2942211 rw_platform 10.92.170.49:55516 platform Sleep 2
2942213 rw_platform 10.92.170.49:55520 platform Sleep 5
2942216 rw_platform 10.92.170.49:55526 platform Sleep 3
2942219 rw_seata 10.92.170.47:34492 seata Sleep 3
2942220 rw_seata 10.92.170.49:55530 seata Sleep 1
2942221 rw_seata 10.92.170.47:34494 seata Sleep 1 shawn 发表于 2023-8-7 10:29
show processlist
10 system user Connect 21428318 waiting for handler commit Group replication a ...
vmstat -S m 1 300
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
rb swpd free buffcache si so bi bo in cs us sy id wa st
10 33 7230 044001 0 0 7 58 0 020 9800
10 33 7227 044002 0 0 02344 24296 2212630 9700
10 33 7228 044002 0 0 01966 18641 1782630 9600
10 33 7227 044002 0 0 03261 18126 1847230 9700
10 33 7227 044003 0 0 02245 18434 1854620 9700
20 33 7227 044003 0 0 01407 14932 1437920 9800
40 33 7226 044003 0 0 02710 18971 1881820 9800
10 33 7226 044003 0 0 02138 17956 1743920 9800
00 33 7225 044004 0 0 03446 23155 2309131 9700
10 33 7226 044004 0 0 03127 22316 2263121 9700
20 33 7225 044004 0 0 01155 10681 1041210 9900
00 33 7225 044005 0 0 01431 13422 1274320 9800
10 33 7225 044004 0 0 01264 15079 1359820 9800
10 33 7225 044005 0 0 01776 18844 1791420 9700
10 33 7224 044005 0 0 02285 19350 1935020 9700
10 33 7225 044005 0 0 01431 23766 2000130 9700
10 33 7225 044005 0 0 0 460 15398 1150420 9800
20 33 7224 044005 0 0 01389 14425 1382720 9700
00 33 7225 044005 0 0 0 41040 18006 1687520 9800
00 33 7225 044006 0 0 02985 23361 2282040 9500
20 33 7225 044006 0 0 03914 16899 1540940 9500
10 33 7224 044006 0 0 03047 15813 1380620 9700
20 33 7224 044006 0 0 02787 12132 1139610 9800
00 33 7224 044006 0 0 03324 13362 1294620 9800
10 33 7224 044006 0 0 03182 14931 1306030 9700 proxysql中closing unhealthy client connection 意思是由于客户端不正常关闭连接导致的从而会记录一条日志，你的mgr是有什么问题么？ doubled 发表于 2023-8-7 10:45
proxysql中closing unhealthy client connection 意思是由于客户端不正常关闭连接导致的从而会记录一条日志 ...
在proxysql中可以设置log_unhealthy_connections=false，可以屏蔽这种日志 doubled 发表于 2023-8-7 10:45
proxysql中closing unhealthy client connection 意思是由于客户端不正常关闭连接导致的从而会记录一条日志 ...
这种日志影响不大，最主要的是探测到MGR的节点存在lag比较频繁。 shawn 发表于 2023-8-7 10:52
这种日志影响不大，最主要的是探测到MGR的节点存在lag比较频繁。
几个方向： doubled 发表于 2023-8-7 10:55
几个方向：
几个方向：1、mgr节点之间是否真的出现网络问题导致的延迟；2、proxysql中日志有没有记录相应的延迟，如果有，对应的日志可以发下；3、proxysql与mgr节点之间网络是否有问题