GreatSQL社区

搜索

[已解决] docker-compose 部署mgr集群 ,mysql-router 路由 主节点无法恢复

398 3 2024-10-16 14:01
1金币

采用 docker-compose 部署mgr集群 ,mysql-router 做高可用 , 主节点无法恢复的问题 。

三台服务器 构建 mgr 一主 、一丛、一仲裁:
172.30.0.2greatsql-master
172.30.0.3 greatsql-slave
172.30.0.4 greatsql-arb



采用docker-compose 部署mgr集群 yml 配置文件内容如下,其实就说官方例子 GreatSQL-Docker 拷贝下来 增加了不同挂载目录 ,防止数据覆盖 。


    1. version: '2'

    2. services:
    3.   greatsql-master:
    4.     #image: greatsql/greatsql        #指定镜像
    5.     image: registry.cn-beijing.aliyuncs.com/greatsql/greatsql        #指定镜像(阿里云ACR)
    6.     container_name: greatsql-master            #设定容器名字
    7.     hostname: greatsql-master                  #设定容器中的主机名
    8.     networks:                       #指定容器使用哪个专用网络
    9.       mgr_net:
    10.         ipv4_address: 172.30.0.2    #设置容器使用固定IP地址,避免重启后IP变化
    11.     restart: unless-stopped         #设定重启策略
    12.     environment:                    #设置多个环境变量
    13.       TZ: Asia/Shanghai             #时区
    14.       MYSQL_ALLOW_EMPTY_PASSWORD: 1                 #允许root账户空密码
    15.       MYSQL_INIT_MGR: 1                             #初始化MGR集群
    16.       #MYSQL_MGR_NAME: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaa1"       #MGR group_name
    17.       #MYSQL_MGR_VIEWID: "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbb1"     #MGR view_change_uuid
    18.       MYSQL_MGR_LOCAL: '172.30.0.2:33061'           #当前MGR节点的local_address
    19.       MYSQL_MGR_SEEDS: '172.30.0.2:33061,172.30.0.3:33061,172.30.0.4:33061'     #MGR集群seeds
    20.       MYSQL_MGR_START_AS_PRIMARY: 1                 #指定当前MGR节点为Primary角色
    21.       MYSQL_MGR_ARBITRATOR: 0                       #指定当前MGR节点为Arbitrator角色,此时不能同时指定其为Primary/Secondary角色
    22.       LOWER_CASE_TABLE_NAMES: 0                     #设定lower_case_table_names值,默认为0
    23.       MAXPERF: 0
    24.     volumes:
    25.       - ./data_master:/data
    26.         # - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime

    27.   greatsql-slave:
    28.     #image: greatsql/greatsql        #指定镜像
    29.     image: registry.cn-beijing.aliyuncs.com/greatsql/greatsql        #指定镜像(阿里云ACR)
    30.     container_name: greatsql-slave
    31.     hostname: greatsql-slave
    32.     networks:
    33.       mgr_net:
    34.         ipv4_address: 172.30.0.3
    35.     restart: unless-stopped
    36.     depends_on:
    37.       - "greatsql-master"
    38.     environment:
    39.       TZ: Asia/Shanghai
    40.       MYSQL_ALLOW_EMPTY_PASSWORD: 1
    41.       MYSQL_INIT_MGR: 1
    42.       MYSQL_MGR_LOCAL: '172.30.0.3:33061'
    43.       MYSQL_MGR_SEEDS: '172.30.0.2:33061,172.30.0.3:33061,172.30.0.4:33061'
    44.       MYSQL_MGR_START_AS_PRIMARY: 0
    45.       MYSQL_MGR_ARBITRATOR: 0                       #既非Primary,也非Arbitrator,那么就是Secondary角色了
    46.       LOWER_CASE_TABLE_NAMES: 0
    47.       MAXPERF: 0
    48.     volumes:
    49.       - ./data_slave:/data
    50.         # - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    51.   greatsql-arb:
    52.     #image: greatsql/greatsql        #指定镜像
    53.     image: registry.cn-beijing.aliyuncs.com/greatsql/greatsql        #指定镜像(阿里云ACR)
    54.     container_name: greatsql-arb
    55.     hostname: greatsql-arb
    56.     networks:
    57.       mgr_net:
    58.         ipv4_address: 172.30.0.4
    59.     restart: unless-stopped
    60.     depends_on:
    61.       - "greatsql-slave"
    62.     environment:
    63.       TZ: Asia/Shanghai
    64.       MYSQL_ALLOW_EMPTY_PASSWORD: 1
    65.       MYSQL_INIT_MGR: 1
    66.       MYSQL_MGR_LOCAL: '172.30.0.4:33061'
    67.       MYSQL_MGR_SEEDS: '172.30.0.2:33061,172.30.0.3:33061,172.30.0.4:33061'
    68.       MYSQL_MGR_ARBITRATOR: 1                   #指定当前MGR节点为Arbitrator角色,此时不能同时指定其为Primary/Secondary角色
    69.       LOWER_CASE_TABLE_NAMES: 0
    70.       MAXPERF: 0
    71.     volumes:
    72.       - ./data_arb:/data
    73.       # - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    74. networks:
    75.   mgr_net:  #创建独立MGR专属网络
    76.     ipam:
    77.       config:
    78.         - subnet: 172.30.0.0/26
    复制代码
    2、部署测试结果如下 :

   
       从节点
greatsql-slave

和仲裁节点
greatsql-arb

都可以在停止后, 再次启动可以子自动恢复到集群 。
      
      部署日志:

     

  1. [+] Building 0.0s (0/0)                                                                                                                                                                        docker:default
  2. [+] Running 4/4
  3. ✔ Network docker-compose_mgr_net  Created                                                                                                                                                               0.2s
  4. ✔ Container greatsql-master       Created                                                                                                                                                               0.7s
  5. ✔ Container greatsql-slave        Created                                                                                                                                                               0.5s
  6. ✔ Container greatsql-arb          Created                                                                                                                                                               0.5s
  7. Attaching to greatsql-arb, greatsql-master, greatsql-slave
  8. greatsql-master  | Initializing database
  9. greatsql-slave   | Initializing database
  10. greatsql-arb     | Initializing database
  11. greatsql-master  | Database initialized
  12. greatsql-master  | MySQL init process in progress...
  13. greatsql-arb     | Database initialized
  14. greatsql-arb     | MySQL init process in progress...
  15. greatsql-master  | MySQL init process in progress...
  16. greatsql-arb     | MySQL init process in progress...
  17. greatsql-master  | MySQL init process in progress...
  18. greatsql-arb     | MySQL init process in progress...
  19. greatsql-master  | MySQL init process in progress...
  20. greatsql-arb     | MySQL init process in progress...
  21. greatsql-master  | MySQL init process in progress...
  22. greatsql-slave   | Database initialized
  23. greatsql-arb     | MySQL init process in progress...
  24. greatsql-slave   | MySQL init process in progress...
  25. greatsql-master  | MySQL init process in progress...
  26. greatsql-arb     | MySQL init process in progress...
  27. greatsql-slave   | MySQL init process in progress...
  28. greatsql-master  | MySQL init process in progress...
  29. greatsql-arb     | MySQL init process in progress...
  30. greatsql-slave   | MySQL init process in progress...
  31. greatsql-master  | MySQL init process in progress...
  32. greatsql-arb     | MySQL init process in progress...
  33. greatsql-slave   | MySQL init process in progress...
  34. greatsql-master  | MySQL init process in progress...
  35. greatsql-arb     | MySQL init process in progress...
  36. greatsql-slave   | MySQL init process in progress...
  37. greatsql-master  | MySQL init process in progress...
  38. greatsql-arb     | MySQL init process in progress...
  39. greatsql-slave   | MySQL init process in progress...
  40. greatsql-master  | MySQL init process in progress...
  41. greatsql-arb     | MySQL init process in progress...
  42. greatsql-slave   | MySQL init process in progress...
  43. greatsql-master  | mysql: [Warning] Using a password on the command line interface can be insecure.
  44. greatsql-arb     | mysql: [Warning] Using a password on the command line interface can be insecure.
  45. greatsql-slave   | MySQL init process in progress...
  46. greatsql-slave   | MySQL init process in progress...
  47. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
  48. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
  49. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
  50. greatsql-slave   | mysql: [Warning] Using a password on the command line interface can be insecure.
  51. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
  52. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
  53. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
  54. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  55. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  56. greatsql-master  | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  57. greatsql-master  | mysql: [Warning] Using a password on the command line interface can be insecure.
  58. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  59. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  60. greatsql-arb     | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  61. greatsql-arb     | mysql: [Warning] Using a password on the command line interface can be insecure.
  62. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
  63. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
  64. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
  65. greatsql-master  |
  66. greatsql-master  | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  67. greatsql-master  |
  68. greatsql-arb     |
  69. greatsql-arb     | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  70. greatsql-arb     |
  71. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  72. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  73. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  74. greatsql-slave   | mysql: [Warning] Using a password on the command line interface can be insecure.
  75. greatsql-slave   |
  76. greatsql-slave   | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  77. greatsql-slave   |
  78. greatsql-master  |
  79. greatsql-master  | MySQL init process done. Ready for start up.
  80. greatsql-master  |
  81. greatsql-arb     |
  82. greatsql-arb     | MySQL init process done. Ready for start up.
  83. greatsql-arb     |
  84. greatsql-arb     | /greatsql-init.sh: line 330: [: -eq: unary operator expected
  85. greatsql-arb     | MAXPERF invalid
  86. greatsql-master  | MAXPERF invalid
  87. greatsql-slave   |
  88. greatsql-slave   | MySQL init process done. Ready for start up.
  89. greatsql-slave   |
  90. greatsql-slave   | MAXPERF invalid
复制代码


进入 greatsql-master 查看整个 mgr 集群情况
  1. docker exec -it greatsql-master bash
  2. bash-4.4# mysql
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 31
  5. Server version: 8.0.32-26 GreatSQL (GPL), Release 26, Revision a68b3034c3d

  6. Copyright (c) 2021-2024 GreatDB Software Co., Ltd
  7. Copyright (c) 2009-2024 Percona LLC and/or its affiliates
  8. Copyright (c) 2000, 2024, Oracle and/or its affiliates.

  9. Oracle is a registered trademark of Oracle Corporation and/or its
  10. affiliates. Other names may be trademarks of their respective
  11. owners.

  12. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

  13. (Wed Oct 16 13:47:13 2024)[root@GreatSQL][(none)]> ^C
  14. ^C
  15. (Wed Oct 16 13:47:23 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  16. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  17. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  18. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  19. | group_replication_applier | 4e79de51-8b81-11ef-a19d-0242ac1e0002 | 172.30.0.2  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  20. | group_replication_applier | 4f937f88-8b81-11ef-a301-0242ac1e0003 | 172.30.0.3  |        3306 | ONLINE       | SECONDARY   | 8.0.32         | XCom                       |
  21. | group_replication_applier | 508fae3b-8b81-11ef-a1fe-0242ac1e0004 | 172.30.0.4  |        3306 | ONLINE       | ARBITRATOR  | 8.0.32         | XCom                       |
  22. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  23. 3 rows in set (0.03 sec)

复制代码


停止 greatsql-master


  1. GreatSQL-Docker/GreatSQL/docker-compose$ docker-compose -f mgr-3nodes.yml stop greatsql-master
  2. [+] Stopping 1/1
  3. ✔ Container greatsql-master  Stopped                                 

复制代码
进入 slave 容器查看情况 :

  1. 发现从节点变成主节点 :
复制代码
  1. 初始化页面可以看到 相关 启动 、停止信息
复制代码




再次启动 greatsql-master
  1. docker-compose -f mgr-3nodes.yml start greatsql-master
  2. [+] Running 1/1
  3. ✔ Container greatsql-master  Started                             
复制代码


  1. greatsql-master  |
  2. greatsql-master  | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  3. greatsql-master  |
  4. greatsql-arb     |
  5. greatsql-arb     | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  6. greatsql-arb     |
  7. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
  8. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
  9. greatsql-slave   | Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
  10. greatsql-slave   | mysql: [Warning] Using a password on the command line interface can be insecure.
  11. greatsql-slave   |
  12. greatsql-slave   | /greatsql-init.sh: ignoring /docker-entrypoint-initdb.d/*
  13. greatsql-slave   |
  14. greatsql-master  |
  15. greatsql-master  | MySQL init process done. Ready for start up.
  16. greatsql-master  |
  17. greatsql-arb     |
  18. greatsql-arb     | MySQL init process done. Ready for start up.
  19. greatsql-arb     |
  20. greatsql-arb     | /greatsql-init.sh: line 330: [: -eq: unary operator expected
  21. greatsql-arb     | MAXPERF invalid
  22. greatsql-master  | MAXPERF invalid
  23. greatsql-slave   |
  24. greatsql-slave   | MySQL init process done. Ready for start up.
  25. greatsql-slave   |
  26. greatsql-slave   | MAXPERF invalid
  27. greatsql-master exited with code 0 关闭退出
  28. greatsql-master  | MAXPERF invalid  启动
  29. 可以看到greatsql-master 启动相关信息 ,
复制代码
并再次查看进入greatesql-slave 节点查看信息 ,发现之前掉线的的主节点没有恢复到mgr集群里面来 。

  1. docker exec -it greatsql-slave bash
  2. bash-4.4# mysql
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 36
  5. Server version: 8.0.32-26 GreatSQL (GPL), Release 26, Revision a68b3034c3d

  6. Copyright (c) 2021-2024 GreatDB Software Co., Ltd
  7. Copyright (c) 2009-2024 Percona LLC and/or its affiliates
  8. Copyright (c) 2000, 2024, Oracle and/or its affiliates.

  9. Oracle is a registered trademark of Oracle Corporation and/or its
  10. affiliates. Other names may be trademarks of their respective
  11. owners.

  12. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

  13. (Wed Oct 16 13:50:49 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  14. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  15. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  16. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  17. | group_replication_applier | 4f937f88-8b81-11ef-a301-0242ac1e0003 | 172.30.0.3  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  18. | group_replication_applier | 508fae3b-8b81-11ef-a1fe-0242ac1e0004 | 172.30.0.4  |        3306 | ONLINE       | ARBITRATOR  | 8.0.32         | XCom                       |
  19. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  20. 2 rows in set (0.09 sec)

  21. (Wed Oct 16 13:50:57 2024)[root@GreatSQL][(none)]> ^C
  22. ^C
  23. (Wed Oct 16 13:54:54 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  24. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  25. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  26. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  27. | group_replication_applier | 4f937f88-8b81-11ef-a301-0242ac1e0003 | 172.30.0.3  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  28. | group_replication_applier | 508fae3b-8b81-11ef-a1fe-0242ac1e0004 | 172.30.0.4  |        3306 | ONLINE       | ARBITRATOR  | 8.0.32         | XCom                       |
  29. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  30. 2 rows in set (0.01 sec)
复制代码
进入 greatesql-master 里面查看集群信息:发现该节点信息任然为 PRIMARY  无法自动更改为从节点 。


  1. docker exec -it greatsql-master bash
  2. bash-4.4# mysql
  3. Welcome to the MySQL monitor.  Commands end with ; or \g.
  4. Your MySQL connection id is 24
  5. Server version: 8.0.32-26 GreatSQL (GPL), Release 26, Revision a68b3034c3d

  6. Copyright (c) 2021-2024 GreatDB Software Co., Ltd
  7. Copyright (c) 2009-2024 Percona LLC and/or its affiliates
  8. Copyright (c) 2000, 2024, Oracle and/or its affiliates.

  9. Oracle is a registered trademark of Oracle Corporation and/or its
  10. affiliates. Other names may be trademarks of their respective
  11. owners.

  12. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

  13. (Wed Oct 16 13:56:37 2024)[root@GreatSQL][(none)]> SELECT * FROM performance_schema.replication_group_members;
  14. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  15. | CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
  16. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  17. | group_replication_applier | 4e79de51-8b81-11ef-a19d-0242ac1e0002 | 172.30.0.2  |        3306 | ONLINE       | PRIMARY     | 8.0.32         | XCom                       |
  18. +---------------------------+--------------------------------------+-------------+-------------+--------------+-------------+----------------+----------------------------+
  19. 1 row in set (0.05 sec)

复制代码


是否有办法解决这种问题 ,因为在宿主机上面部署mgr集群 、后续升级维也特别麻烦 ;如果采用K8S 部署项目条件达不到 。
请问采用docker-compose 部署这种方案是否有办法解决这种问题 。

全部回复(3)
驭无殇1998 2024-10-16 14:12:47
如果你只是测试环境使用docker部署个人感觉是没问题的。

但是生产也使用docker的方式进行部署的话,不太推荐,还是建议直接部署到宿主机比较好维护,关于升级的话,如果是mgr集群,不是修改特别大的版本升级的话,直接替换安装包,滚动升级的方式也是很方便的
yejr 2024-10-16 15:18:02
GreatSQL docker镜像的逻辑是只支持预设指定主从节点,不支持切换后继续保持新的主从关系。也即利用docker-compose构建好一个MGR集群后,只能整体重启后还维持预设指定的拓扑关系,不支持在运行过程中发生主从切换后,再次重启还能继续维持这个主从拓扑。

具体的处理逻辑见 https://gitee.com/GreatSQL/Great ... QL/greatsql-init.sh

如果需要实现上述需求得自己重新封装处理哈。
openai 2024-10-16 15:19:59
驭无殇1998 发表于 2024-10-16 14:12
如果你只是测试环境使用docker部署个人感觉是没问题的。

但是生产也使用docker的方式进行部署的话,不太推 ...

非常感谢
openai

3

主题

0

博客

7

贡献

新手上路

Rank: 1

积分
11

合作电话:010-64087828

社区邮箱:greatsql@greatdb.com

社区公众号
社区小助手
QQ群
GMT+8, 2024-12-4 20:52 , Processed in 0.022177 second(s), 15 queries , Redis On.
快速回复 返回顶部 返回列表