centos7.6部署mongodb复制集RS（ReplicationSet），通过多实例实现，即一台服务器部署4个实例，通过不同的端口区分。
MongoDB中的复制集(也被称为副本)是一组维护相同数据集的mongod进程。副本集提供冗余性及和高可用，是所有生产部署的基础。简单来说，复制集有多台MongoDB组成的一个集群，集群中有一个主节点(Primary)和N个副本节点(Secondary)等，它们有相同的数据库，假如主MongoDB服务器或者MongoDB实例Down机之后，其它的副本服务器可以继续提供服务，实现数据的高可用及可靠性。
1 mongodb复制集架构
1.1 角色介绍：
术语 类型 描述
Primary 主节点 负责整个集群的读写操作，包含了所有改变操作的日志
Secondary 备节点 同步主服务器所有的数据，负责集群的读取请求，主服务器宕机可以称为主节点
Arbiter 仲裁者 在主节点宕机后只进行投票，不参与选举，不同步主节点数据
1.2 mongodb的复制集架构主要分为两种，普通复制集和选主复制集。
1、普通复制集架构：
image.png
主节点故障后，复制集变化过程
image.png
2、选主复制集架构（仲裁节点架构）
image.png
主节点故障后，复制集变化过程
image.png
1.3 基本原理
复制集基本架构为1主2从或1主1从1Arbiter 。而且mongodb复制集自带互相监控投票机制，其内部算法为【MongoD(Raft)】和【Paxos(mysql MGR )】算法基本相同。
如果发生主库宕机，复制集内部会根据Raft算法进行投票选举，选择一个新的主库替代原有主库对外提供服务。同时复制集会自动通知客户端程序，主库已经发生切换了，应用就会连接到新的主库。
2 复制集架构规划
可以使用3个节点或多实例实现。本案例通过多实例实现。
2.1 多实例规划
节点:172.21.209.123
端口：28017-28020（实现多实例）
2.2 多实例目录
登陆到mongod用户并创建多实例目录
[root@mysql-master ~]# su - mongod
[mongod@mysql-master /data]$ mkdir /data/mongodb/mult-mongodb
2.3 创建多实例数据目录
[mongod@mysql-master mult-mongodb]$ for path in {28017..28020}
> do
> mkdir -vp /data/mongodb/mult-mongodb/$path/{data,conf,log}
> done
mkdir: created directory ‘/data/mongodb/mult-mongodb/28017’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28017/data’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28017/conf’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28017/log’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28018’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28018/data’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28018/conf’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28018/log’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28019’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28019/data’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28019/conf’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28019/log’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28020’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28020/data’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28020/conf’
mkdir: created directory ‘/data/mongodb/mult-mongodb/28020/log’
查看创建的多实例目录，通过不同的端口区分
[mongod@mysql-master mult-mongodb]$ ll
total 0
drwxrwxr-x 5 mongod mongod 41 Sep 24 04:52 28017
drwxrwxr-x 5 mongod mongod 41 Sep 24 04:52 28018
drwxrwxr-x 5 mongod mongod 41 Sep 24 04:52 28019
drwxrwxr-x 5 mongod mongod 41 Sep 24 04:52 28020
2.4 多实例配置文件
1、配置文件路径
/data/mongodb/mult-mongodb/28017/conf/mongod.conf
/data/mongodb/mult-mongodb/28018/conf/mongod.conf
/data/mongodb/mult-mongodb/28019/conf/mongod.conf
/data/mongodb/mult-mongodb/28020/conf/mongod.conf
2、配置文件内容
cat >/data/mongodb/mult-mongodb/28017/conf/mongod.conf <<EOF
systemLog:
destination: file
path: /data/mongodb/mult-mongodb/28017/log/mongodb.log
logAppend: true
storage:
journal:
enabled: true
dbPath: /data/mongodb/mult-mongodb/28017/data
directoryPerDB: true
#engine: wiredTiger
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
net:
bindIp: 172.21.209.123,127.0.0.1
port: 28017
replication:
oplogSizeMB: 2048
replSetName: myReplSet
EOF
3、复制配置 文件到其他实例的conf目录下，并修改端口
[mongod@mysql-master conf]$ cp /data/mongodb/mult-mongodb/28017/conf/mongod.conf /data/mongodb/mult-mongodb/28018/conf/
[mongod@mysql-master conf]$ cp /data/mongodb/mult-mongodb/28017/conf/mongod.conf /data/mongodb/mult-mongodb/28019/conf/
[mongod@mysql-master conf]$ cp /data/mongodb/mult-mongodb/28017/conf/mongod.conf /data/mongodb/mult-mongodb/28020/conf/
修改配置文件
[mongod@mysql-master conf]$ sed 's#28017#28018#g' /data/mongodb/mult-mongodb/28018/conf/mongod.conf -i
[mongod@mysql-master conf]$ sed 's#28017#28019#g' /data/mongodb/mult-mongodb/28019/conf/mongod.conf -i
[mongod@mysql-master conf]$ sed 's#28017#28020#g' /data/mongodb/mult-mongodb/28020/conf/mongod.conf -i
2.5 启动多实例
mongod -f /data/mongodb/mult-mongodb/28017/conf/mongod.conf
mongod -f /data/mongodb/mult-mongodb/28018/conf/mongod.conf
mongod -f /data/mongodb/mult-mongodb/28019/conf/mongod.conf
mongod -f /data/mongodb/mult-mongodb/28020/conf/mongod.conf
启动过程：
[mongod@mysql-master conf]$ mongod -f /data/mongodb/mult-mongodb/28017/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 29035
child process started successfully, parent exiting
[mongod@mysql-master conf]$ mongod -f /data/mongodb/mult-mongodb/28018/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 29073
child process started successfully, parent exiting
[mongod@mysql-master conf]$ mongod -f /data/mongodb/mult-mongodb/28019/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 29111
child process started successfully, parent exiting
[mongod@mysql-master conf]$ mongod -f /data/mongodb/mult-mongodb/28020/conf/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 29149
child process started successfully, parent exiting
2.6 查看进程和端口
[mongod@mysql-master conf]$ ps -ef|grep mongodb
[mongod@mysql-master conf]$ netstat -lntp|grep 280
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 172.21.209.123:28017 0.0.0.0:* LISTEN 29035/mongod
tcp 0 0 127.0.0.1:28017 0.0.0.0:* LISTEN 29035/mongod
tcp 0 0 172.21.209.123:28018 0.0.0.0:* LISTEN 29073/mongod
tcp 0 0 127.0.0.1:28018 0.0.0.0:* LISTEN 29073/mongod
tcp 0 0 172.21.209.123:28019 0.0.0.0:* LISTEN 29111/mongod
tcp 0 0 127.0.0.1:28019 0.0.0.0:* LISTEN 29111/mongod
tcp 0 0 172.21.209.123:28020 0.0.0.0:* LISTEN 29149/mongod
tcp 0 0 127.0.0.1:28020 0.0.0.0:* LISTEN 29149/mongod
3 配置复制集
3.1 配置普通复制集【1主2从】
1、登陆到mongodb数据库
[mongod@mysql-master data]$ mongo --port 28017 admin
2、配置mongodb复制集
config = {_id: 'myReplSet', members: [
{_id: 0, host: '172.21.209.123:28017'},
{_id: 1, host: '172.21.209.123:28018'},
{_id: 2, host: '172.21.209.123:28019'}]
}
rs.initiate(config)
注：
admin为验证库。
集群过程中会出现两种角色PRIMARY和SECONDARY，为配置成功。
myReplSet:SECONDARY>
myReplSet:PRIMARY>
3、查看复制集集群的状态
myReplSet:PRIMARY> rs.status()
3.2、配置选择复制集【1主1从1个arbiter】
[mongod@mysql-master data]$ mongo --port 28017 admin
config = {_id: 'myReplSet', members: [
{_id: 0, host: '172.21.209.123:28017'},
{_id: 1, host: '172.21.209.123:28018'},
{_id: 2, host: '172.21.209.123:28019',"arbiterOnly":true}]
}
rs.initiate(config)
注：myReplSet要和配置文件中的复制集模块一致。
arbiterOnly设置为true.
3.3、复制集常用管理操作
rs.status(); //查看复制集状态
rs.isMaster(); // 查看当前是否为主节点
rs.conf()； //查看复制集配置信息
1、查看复制集状态
myReplSet:PRIMARY> rs.status()
{
"set" : "myReplSet",
"date" : ISODate("2021-09-24T09:35:30.813Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2021-09-24T09:35:27.923Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2021-09-24T09:35:27.923Z"),
"appliedOpTime" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"lastAppliedWallTime" : ISODate("2021-09-24T09:35:27.923Z"),
"lastDurableWallTime" : ISODate("2021-09-24T09:35:27.923Z")
},
"lastStableRecoveryTimestamp" : Timestamp(1632476117, 1),
"lastStableCheckpointTimestamp" : Timestamp(1632476117, 1),
"electionCandidateMetrics" : {
"lastElectionReason" : "electionTimeout",
"lastElectionDate" : ISODate("2021-09-24T09:21:17.879Z"),
"electionTerm" : NumberLong(1),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1632475267, 1),
"t" : NumberLong(-1)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2021-09-24T09:21:17.905Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2021-09-24T09:21:18.701Z")
},
"members" : [
{
"_id" : 0,
"name" : "172.21.209.123:28017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 1645,
"optime" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2021-09-24T09:35:27Z"),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"infoMessage" : "",
"electionTime" : Timestamp(1632475277, 1),
"electionDate" : ISODate("2021-09-24T09:21:17Z"),
"configVersion" : 1,
"self" : true,
"lastHeartbeatMessage" : ""
},
{
"_id" : 1,
"name" : "172.21.209.123:28018",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 863,
"optime" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2021-09-24T09:35:27Z"),
"optimeDurableDate" : ISODate("2021-09-24T09:35:27Z"),
"lastHeartbeat" : ISODate("2021-09-24T09:35:29.897Z"),
"lastHeartbeatRecv" : ISODate("2021-09-24T09:35:29.171Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "172.21.209.123:28017",
"syncSourceHost" : "172.21.209.123:28017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 1
},
{
"_id" : 2,
"name" : "172.21.209.123:28019",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 863,
"optime" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"optimeDurable" : {
"ts" : Timestamp(1632476127, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2021-09-24T09:35:27Z"),
"optimeDurableDate" : ISODate("2021-09-24T09:35:27Z"),
"lastHeartbeat" : ISODate("2021-09-24T09:35:29.896Z"),
"lastHeartbeatRecv" : ISODate("2021-09-24T09:35:30.673Z"),
"pingMs" : NumberLong(0),
"lastHeartbeatMessage" : "",
"syncingTo" : "172.21.209.123:28017",
"syncSourceHost" : "172.21.209.123:28017",
"syncSourceId" : 0,
"infoMessage" : "",
"configVersion" : 1
}
],
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1632476127, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1632476127, 1)
}
myReplSet:PRIMARY>
2、查看当前节点是否为主节点
myReplSet:PRIMARY> rs.isMaster()
{
"hosts" : [
"172.21.209.123:28017",
"172.21.209.123:28018",
"172.21.209.123:28019"
],
"setName" : "myReplSet",
"setVersion" : 1,
"ismaster" : true,
"secondary" : false,
"primary" : "172.21.209.123:28017",
"me" : "172.21.209.123:28017",
"electionId" : ObjectId("7fffffff0000000000000001"),
"lastWrite" : {
"opTime" : {
"ts" : Timestamp(1632476167, 1),
"t" : NumberLong(1)
},
"lastWriteDate" : ISODate("2021-09-24T09:36:07Z"),
"majorityOpTime" : {
"ts" : Timestamp(1632476167, 1),
"t" : NumberLong(1)
},
"majorityWriteDate" : ISODate("2021-09-24T09:36:07Z")
},
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 100000,
"localTime" : ISODate("2021-09-24T09:36:10.629Z"),
"logicalSessionTimeoutMinutes" : 30,
"connectionId" : 6,
"minWireVersion" : 0,
"maxWireVersion" : 8,
"readOnly" : false,
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1632476167, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1632476167, 1)
}
myReplSet:PRIMARY>
3、查看复制集配置信息
myReplSet:PRIMARY> rs.conf()
{
"_id" : "myReplSet",
"version" : 1,
"protocolVersion" : NumberLong(1),
"writeConcernMajorityJournalDefault" : true,
"members" : [
{
"_id" : 0,
"host" : "172.21.209.123:28017",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 1,
"host" : "172.21.209.123:28018",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
},
{
"_id" : 2,
"host" : "172.21.209.123:28019",
"arbiterOnly" : false,
"buildIndexes" : true,
"hidden" : false,
"priority" : 1,
"tags" : {
},
"slaveDelay" : NumberLong(0),
"votes" : 1
}
],
"settings" : {
"chainingAllowed" : true,
"heartbeatIntervalMillis" : 2000,
"heartbeatTimeoutSecs" : 10,
"electionTimeoutMillis" : 10000,
"catchUpTimeoutMillis" : -1,
"catchUpTakeoverDelayMillis" : 30000,
"getLastErrorModes" : {
},
"getLastErrorDefaults" : {
"w" : 1,
"wtimeout" : 0
},
"replicaSetId" : ObjectId("614d98833680adfa6bcd2c0e")
}
}
myReplSet:PRIMARY>
4 复制集节点的添加和删除
1、添加删除常用操作
rs.remove("节点ip:port"); // 删除一个节点
rs.add("节点ip:port"); // 新增从节点
rs.addArb("节点ip:port"); // 新增仲裁节点
2、案例操作
连接到主节点
[mongod@mysql-master data]$ mongo --port 28017 admin
2.1 添加仲裁节点
myReplSet:PRIMARY> rs.addArb("172.21.209.123:28020")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1632476954, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1632476954, 1)
}
添加后查看状态：myReplSet:PRIMARY> rs.isMaster()
2.2 删除一个节点
myReplSet:PRIMARY> rs.remove("172.21.209.123:28019");
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1632477110, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1632477110, 1)
}
删除后查看状态：myReplSet:PRIMARY> rs.isMaster()
2.3 新增加从节点
myReplSet:PRIMARY> rs.add("1172.21.209.123:28019")
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1632477186, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1632477186, 1)
}
新增后查看状态：myReplSet:PRIMARY> rs.isMaster()
5 复制集的特殊节点
5.1 介绍和架构
- arbiter节点：主要负责选主过程投票，但不存储任何数据，也不向客户端提供任何服务
- hidden节点：隐藏节点，不参与选主，也不向客户端提供任何服务。
- delay节点：延时节点，数据落后于主库一段时间，因为数据是延时的，也不应该提供服务或参与选主，所以通常会配合hidden（隐藏）
一般情况下会将delay+hidden一起配置使用
image.png
image.png
5.2 配置延时节点（组合使用：delay+hidden）
1、配置延时
cfg=rs.conf()
cfg.members[1].priority=0
cfg.members[1].hidden=true
cfg.members[1].slaveDelay=120
rs.reconfig(cfg)
2、恢复延时配置
cfg=rs.conf()
cfg.members[1].priority=1
cfg.members[1].hidden=false
cfg.members[1].slaveDelay=0
rs.reconfig(cfg)
3、配置成功后，通过以下命令查询配置后的属性
rs.conf();
注：通过rs.status()可以查看到members[1]是一个数组，下标从0开始，选择需要配置的节点，members[1]表示第三个节点。
6 复制集其他命令
1、查看副本集的配置信息
myReplSet:PRIMARY> rs.conf()
2、查看副本集各成员的状态
myReplSet:PRIMARY> rs.status()
3、副本集角色切换（不要人为随便操作）
myReplSet:PRIMARY> rs.stepDown()
注：
myReplSet:PRIMARY> rs.freeze(300) //锁定从，使其不会转变成主库
freeze()和stepDown单位都是秒。
4、设置副本节点可读：在副本节点执行
登陆从节点
[mongod@mysql-master data]$ mongo --port 28018 admin
myReplSet:SECONDARY> rs.slaveOk()
eg：
myReplSet:SECONDARY> use student;
switched to db student
myReplSet:SECONDARY> db.createCollection('a')
{
"operationTime" : Timestamp(1632478227, 1),
"ok" : 0,
"errmsg" : "not master",
"code" : 10107,
"codeName" : "NotWritablePrimary",
"$clusterTime" : {
"clusterTime" : Timestamp(1632478227, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
5、查看副本节点（监控主从延时）
myReplSet:SECONDARY> rs.printSlaveReplicationInfo()
WARNING: printSlaveReplicationInfo is deprecated and may be removed in the next major release. Please use printSecondaryReplicationInfo instead.
source: 172.21.209.123:28018
syncedTo: Fri Sep 24 2021 06:13:08 GMT-0400 (EDT)
0 secs (0 hrs) behind the primary
source: 1172.21.209.123:28019
syncedTo: Wed Dec 31 1969 19:00:00 GMT-0500 (EST)
1632478388 secs (453466.22 hrs) behind the primary
至此：mongodb的复制集部署完成，通过多实例实现部署。
