点击参与 GreatSQL 8.0.32-26 文档捉虫~~~

GreatSQL社区

搜索

[已解决] docker-compose 部署mgr集群 ，mysql-router 路由 主节点无法恢复

391 3 2024-10-16 14:01
1金币

采用 docker-compose 部署mgr集群 ，mysql-router 做高可用 ， 主节点无法恢复的问题 。

三台服务器 构建 mgr 一主 、一丛、一仲裁：
172.30.0.2greatsql-master
172.30.0.3 greatsql-slave
172.30.0.4 greatsql-arb



采用docker-compose 部署mgr集群 yml 配置文件内容如下，其实就说官方例子 GreatSQL-Docker 拷贝下来 增加了不同挂载目录 ，防止数据覆盖 。


    1. version: '2'

    3. services:
    4.   greatsql-master:
    5.     #image: greatsql/greatsql        #指定镜像
    6.     image: registry.cn-beijing.aliyuncs.com/greatsql/greatsql        #指定镜像（阿里云ACR）
    7.     container_name: greatsql-master            #设定容器名字
    8.     hostname: greatsql-master                  #设定容器中的主机名
    9.     networks:                       #指定容器使用哪个专用网络
    10.       mgr_net:
    11.         ipv4_address: 172.30.0.2    #设置容器使用固定IP地址，避免重启后IP变化
    12.     restart: unless-stopped         #设定重启策略
    13.     environment:                    #设置多个环境变量
    14.       TZ: Asia/Shanghai             #时区
    15.       MYSQL_ALLOW_EMPTY_PASSWORD: 1                 #允许root账户空密码
    16.       MYSQL_INIT_MGR: 1                             #初始化MGR集群
    17.       #MYSQL_MGR_NAME: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1"       #MGR group_name
    18.       #MYSQL_MGR_VIEWID: "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbb1"     #MGR view_change_uuid
    19.       MYSQL_MGR_LOCAL: '172.30.0.2:33061'           #当前MGR节点的local_address
    20.       MYSQL_MGR_SEEDS: '172.30.0.2:33061,172.30.0.3:33061,172.30.0.4:33061'     #MGR集群seeds
    21.       MYSQL_MGR_START_AS_PRIMARY: 1                 #指定当前MGR节点为Primary角色
    22.       MYSQL_MGR_ARBITRATOR: 0                       #指定当前MGR节点为Arbitrator角色，此时不能同时指定其为Primary/Secondary角色
    23.       LOWER_CASE_TABLE_NAMES: 0                     #设定lower_case_table_names值，默认为0
    24.       MAXPERF: 0
    25.     volumes:
    26.       - ./data_master:/data
    27.         # - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime

    29.   greatsql-slave:
    30.     #image: greatsql/greatsql        #指定镜像
    31.     image: registry.cn-beijing.aliyuncs.com/greatsql/greatsql        #指定镜像（阿里云ACR）
    32.     container_name: greatsql-slave
    33.     hostname: greatsql-slave
    34.     networks:
    35.       mgr_net:
    36.         ipv4_address: 172.30.0.3
    37.     restart: unless-stopped
    38.     depends_on:
    39.       - "greatsql-master"
    40.     environment:
    41.       TZ: Asia/Shanghai
    42.       MYSQL_ALLOW_EMPTY_PASSWORD: 1
    43.       MYSQL_INIT_MGR: 1
    44.       MYSQL_MGR_LOCAL: '172.30.0.3:33061'
    45.       MYSQL_MGR_SEEDS: '172.30.0.2:33061,172.30.0.3:33061,172.30.0.4:33061'
    46.       MYSQL_MGR_START_AS_PRIMARY: 0
    47.       MYSQL_MGR_ARBITRATOR: 0                       #既非Primary，也非Arbitrator，那么就是Secondary角色了
    48.       LOWER_CASE_TABLE_NAMES: 0
    49.       MAXPERF: 0
    50.     volumes:
    51.       - ./data_slave:/data
    52.         # - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    53.   greatsql-arb:
    54.     #image: greatsql/greatsql        #指定镜像
    55.     image: registry.cn-beijing.aliyuncs.com/greatsql/greatsql        #指定镜像（阿里云ACR）
    56.     container_name: greatsql-arb
    57.     hostname: greatsql-arb
    58.     networks:
    59.       mgr_net:
    60.         ipv4_address: 172.30.0.4
    61.     restart: unless-stopped
    62.     depends_on:
    63.       - "greatsql-slave"
    64.     environment:
    65.       TZ: Asia/Shanghai
    66.       MYSQL_ALLOW_EMPTY_PASSWORD: 1
    67.       MYSQL_INIT_MGR: 1
    68.       MYSQL_MGR_LOCAL: '172.30.0.4:33061'
    69.       MYSQL_MGR_SEEDS: '172.30.0.2:33061,172.30.0.3:33061,172.30.0.4:33061'
    70.       MYSQL_MGR_ARBITRATOR: 1                   #指定当前MGR节点为Arbitrator角色，此时不能同时指定其为Primary/Secondary角色
    71.       LOWER_CASE_TABLE_NAMES: 0
    72.       MAXPERF: 0
    73.     volumes:
    74.       - ./data_arb:/data
    75.       # - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    76. networks:
    77.   mgr_net:  #创建独立MGR专属网络
    78.     ipam:
    79.       config:
    80.         - subnet: 172.30.0.0/26
    复制代码
    2、部署测试结果如下 ：

   
       从节点
greatsql-slave

和仲裁节点
greatsql-arb

都可以在停止后, 再次启动可以子自动恢复到集群 。
      
      部署日志：

     

  2. [+] Building 0.0s (0/0)                                                                                                                                                                        docker:default
  3. [+] Running 4/4
  4. ✔ Network docker-compose_mgr_net  Created                                                                                                                                                               0.2s
  5. ✔ Container greatsql-master       Created                                                                                                                                                               0.7s
  6. ✔ Container greatsql-slave        Created                                                                                                                                                               0.5s
  7. ✔ Container greatsql-arb          Created                                                                                                                                                               0.5s
  8. Attaching to greatsql-arb, greatsql-master, greatsql-slave
  9. greatsql-master  | Initializing database
  10. greatsql-slave   | Initializing database
  11. greatsql-arb     | Initializing database
  12. greatsql-master  | Database initialized
  13. greatsql-master  | MySQL init process in progress...
  14. greatsql-arb     | Database initialized
  15. greatsql-arb     | MySQL init process in progress...
  16. greatsql-master  | MySQL init process in progress...
  17. greatsql-arb     | MySQL init process in progress...
  18. greatsql-master  | MySQL init process in progress...
  19. greatsql-arb     | MySQL init process in progress...
  20. greatsql-master  | MySQL init process in progress...
  21. greatsql-arb     | MySQL init process in progress...
  22. greatsql-master  | MySQL init process in progress...
  23. greatsql-slave   | Database initialized
  24. greatsql-arb     | MySQL init process in progress...
  25. greatsql-slave   | MySQL init process in progress...
  26. greatsql-master  | MySQL init process in progress...
  27. greatsql-arb     | MySQL init process in progress...
  28. greatsql-slave   | MySQL init process in progress...
  29. greatsql-master  | MySQL init process in progress...
  30. greatsql-arb     | MySQL init process in progress...
  31. greatsql-slave   | MySQL init process in progress...
  32. greatsql-master  | MySQL init process in progress...
  33. greatsql-arb     | MySQL init process in progress...
  34. greatsql-slave   | MySQL init process in progress...
  35. greatsql-master  | MySQL init process in progress...
  36. greatsql-arb     | MySQL init process in progress...
  37. greatsql-slave   | MySQL init process in progress...
  38. greatsql-master  | MySQL init process in progress...
  39. greatsql-arb     | MySQL init process in progress...
  40. greatsql-slave   | MySQL init process in progress...
  41. greatsql-master  | MySQL init process in progress...
  42. greatsql-arb     | MySQL init process in progress...
  43. greatsql-slave   | MySQL init process in progress...
  44. greatsql-master  | mysql: [Warning] Using a password on the command line interface can be insecure.
  45. greatsql-arb     | mysql: [Warning] Using a password on the command line interface can be insecure.
  46. greatsql-slave   | MySQL init process in progress...
  47. greatsql-slave   | MySQL init process in progress...
  48. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
  49. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
  50. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
  51. greatsql-slave   | mysql: [Warning] Using a password on the command line interface can be insecure.
  52. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
  53. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
  54. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
  55. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  56. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  57. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  58. greatsql-master  | mysql: [Warning] Using a password on the command line interface can be insecure.
  59. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  60. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  61. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  62. greatsql-arb     | mysql: [Warning] Using a password on the command line interface can be insecure.
  63. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
  64. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
  65. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
  66. greatsql-master  |
  67. greatsql-master  | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  68. greatsql-master  |
  69. greatsql-arb     |
  70. greatsql-arb     | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  71. greatsql-arb     |
  72. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  73. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  74. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  75. greatsql-slave   | mysql: [Warning] Using a password on the command line interface can be insecure.
  76. greatsql-slave   |
  77. greatsql-slave   | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  78. greatsql-slave   |
  79. greatsql-master  |
  80. greatsql-master  | MySQL init process done. Ready for start up.
  81. greatsql-master  |
  82. greatsql-arb     |
  83. greatsql-arb     | MySQL init process done. Ready for start up.
  84. greatsql-arb     |
  85. greatsql-arb     | /greatsql-init.sh: line 330: [: -eq: unary operator expected
  86. greatsql-arb     | MAXPERF invalid
  87. greatsql-master  | MAXPERF invalid
  88. greatsql-slave   |
  89. greatsql-slave   | MySQL init process done. Ready for start up.
  90. greatsql-slave   |
  91. greatsql-slave   | MAXPERF invalid
复制代码


进入 greatsql-master 查看整个 mgr 集群情况
  1. docker exec -it greatsql-master bash
  2. bash-4.4# mysql
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 31
  5. Server version: 8.0.32-26 GreatSQL (GPL), Release 26, Revision a68b3034c3d

  7. Copyright (c) 2021-2024 GreatDB Software Co., Ltd
  8. Copyright (c) 2009-2024 Percona LLC and/or its affiliates
  9. Copyright (c) 2000, 2024, Oracle and/or its affiliates.

  11. Oracle is a registered trademark of Oracle Corporation and/or its
  12. affiliates. Other names may be trademarks of their respective
  13. owners.

  15. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

  17. (Wed Oct 16 13:47:13 2024)[root@GreatSQL][(none)]> ^C
  18. ^C
  19. (Wed Oct 16 13:47:23 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  20. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  21. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  22. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  23. | group_replication_applier | 4e79de51-8b81-11ef-a19d-0242ac1e0002 | 172.30.0.2  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  24. | group_replication_applier | 4f937f88-8b81-11ef-a301-0242ac1e0003 | 172.30.0.3  |        3306 | ONLINE       | SECONDARY   | 8.0.32         | XCom                       |
  25. | group_replication_applier | 508fae3b-8b81-11ef-a1fe-0242ac1e0004 | 172.30.0.4  |        3306 | ONLINE       | ARBITRATOR  | 8.0.32         | XCom                       |
  26. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  27. 3 rows in set (0.03 sec)

复制代码


停止 greatsql-master


  1. GreatSQL-Docker/GreatSQL/docker-compose$ docker-compose -f mgr-3nodes.yml stop greatsql-master
  2. [+] Stopping 1/1
  3. ✔ Container greatsql-master  Stopped                                 

复制代码
进入 slave 容器查看情况 ：

  1. 发现从节点变成主节点 ：
复制代码
  1. 初始化页面可以看到 相关 启动 、停止信息
复制代码




再次启动 greatsql-master
  1. docker-compose -f mgr-3nodes.yml start greatsql-master
  2. [+] Running 1/1
  3. ✔ Container greatsql-master  Started                             
复制代码


  2. greatsql-master  |
  3. greatsql-master  | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  4. greatsql-master  |
  5. greatsql-arb     |
  6. greatsql-arb     | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  7. greatsql-arb     |
  8. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  9. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  10. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  11. greatsql-slave   | mysql: [Warning] Using a password on the command line interface can be insecure.
  12. greatsql-slave   |
  13. greatsql-slave   | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  14. greatsql-slave   |
  15. greatsql-master  |
  16. greatsql-master  | MySQL init process done. Ready for start up.
  17. greatsql-master  |
  18. greatsql-arb     |
  19. greatsql-arb     | MySQL init process done. Ready for start up.
  20. greatsql-arb     |
  21. greatsql-arb     | /greatsql-init.sh: line 330: [: -eq: unary operator expected
  22. greatsql-arb     | MAXPERF invalid
  23. greatsql-master  | MAXPERF invalid
  24. greatsql-slave   |
  25. greatsql-slave   | MySQL init process done. Ready for start up.
  26. greatsql-slave   |
  27. greatsql-slave   | MAXPERF invalid
  28. greatsql-master exited with code 0 关闭退出
  29. greatsql-master  | MAXPERF invalid  启动
  30. 可以看到greatsql-master 启动相关信息 ，
复制代码
并再次查看进入greatesql-slave 节点查看信息 ，发现之前掉线的的主节点没有恢复到mgr集群里面来 。

  1. docker exec -it greatsql-slave bash
  2. bash-4.4# mysql
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 36
  5. Server version: 8.0.32-26 GreatSQL (GPL), Release 26, Revision a68b3034c3d

  7. Copyright (c) 2021-2024 GreatDB Software Co., Ltd
  8. Copyright (c) 2009-2024 Percona LLC and/or its affiliates
  9. Copyright (c) 2000, 2024, Oracle and/or its affiliates.

  11. Oracle is a registered trademark of Oracle Corporation and/or its
  12. affiliates. Other names may be trademarks of their respective
  13. owners.

  15. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

  17. (Wed Oct 16 13:50:49 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  18. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  19. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  20. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  21. | group_replication_applier | 4f937f88-8b81-11ef-a301-0242ac1e0003 | 172.30.0.3  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  22. | group_replication_applier | 508fae3b-8b81-11ef-a1fe-0242ac1e0004 | 172.30.0.4  |        3306 | ONLINE       | ARBITRATOR  | 8.0.32         | XCom                       |
  23. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  24. 2 rows in set (0.09 sec)

  26. (Wed Oct 16 13:50:57 2024)[root@GreatSQL][(none)]> ^C
  27. ^C
  28. (Wed Oct 16 13:54:54 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  29. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  30. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  31. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  32. | group_replication_applier | 4f937f88-8b81-11ef-a301-0242ac1e0003 | 172.30.0.3  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  33. | group_replication_applier | 508fae3b-8b81-11ef-a1fe-0242ac1e0004 | 172.30.0.4  |        3306 | ONLINE       | ARBITRATOR  | 8.0.32         | XCom                       |
  34. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  35. 2 rows in set (0.01 sec)
复制代码
进入 greatesql-master 里面查看集群信息：发现该节点信息任然为 PRIMARY  无法自动更改为从节点 。


  1. docker exec -it greatsql-master bash
  2. bash-4.4# mysql
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 24
  5. Server version: 8.0.32-26 GreatSQL (GPL), Release 26, Revision a68b3034c3d

  7. Copyright (c) 2021-2024 GreatDB Software Co., Ltd
  8. Copyright (c) 2009-2024 Percona LLC and/or its affiliates
  9. Copyright (c) 2000, 2024, Oracle and/or its affiliates.

  11. Oracle is a registered trademark of Oracle Corporation and/or its
  12. affiliates. Other names may be trademarks of their respective
  13. owners.

  15. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

  17. (Wed Oct 16 13:56:37 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  18. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  19. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  20. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  21. | group_replication_applier | 4e79de51-8b81-11ef-a19d-0242ac1e0002 | 172.30.0.2  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  22. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  23. 1 row in set (0.05 sec)

复制代码


是否有办法解决这种问题 ，因为在宿主机上面部署mgr集群 、后续升级维也特别麻烦 ；如果采用K8S 部署项目条件达不到 。
请问采用docker-compose 部署这种方案是否有办法解决这种问题 。

MGR, GreatSQL, 高可用
全部回复(3)
只看楼主
驭无殇1998 2024-10-16 14:12:47
如果你只是测试环境使用docker部署个人感觉是没问题的。

但是生产也使用docker的方式进行部署的话，不太推荐，还是建议直接部署到宿主机比较好维护，关于升级的话，如果是mgr集群，不是修改特别大的版本升级的话，直接替换安装包，滚动升级的方式也是很方便的
回复 采纳

举报

yejr 2024-10-16 15:18:02
GreatSQL docker镜像的逻辑是只支持预设指定主从节点，不支持切换后继续保持新的主从关系。也即利用docker-compose构建好一个MGR集群后，只能整体重启后还维持预设指定的拓扑关系，不支持在运行过程中发生主从切换后，再次重启还能继续维持这个主从拓扑。

具体的处理逻辑见 https://gitee.com/GreatSQL/Great ... QL/greatsql-init.sh 。

如果需要实现上述需求得自己重新封装处理哈。
回复 已采纳

举报

openai 2024-10-16 15:19:59
驭无殇1998 发表于 2024-10-16 14:12
如果你只是测试环境使用docker部署个人感觉是没问题的。

但是生产也使用docker的方式进行部署的话，不太推 ...

非常感谢
回复 采纳

举报

openai

3

主题

0

博客

7

贡献

新手上路

Rank: 1

积分
11
发消息
互动与交流
社区活动
论坛问答
加入社区
贡献者指南
学习与应用
文章博客
文档资料
视频资料
常见问题
发现社区
社区准则
建议反馈
社区之星

合作电话：010-64087828

社区邮箱：greatsql@greatdb.com

社区公众号
社区小助手
QQ群
友情链接
万里数据库开源中国墨天轮开放原子开源基金会OpenEuler
©2022 GreatSQL All Rights Reserved. ( 京ICP备06057874号 )
GMT+8, 2024-11-21 21:32 , Processed in 0.023661 second(s), 15 queries , Redis On.
快速回复 返回顶部 返回列表