MongoDB集群：分片+副本集

一、环境准备

centos-7.5
服务器：若干
安装包：mongodb-linux-x86_64-3.4.6.tgz

1、架构

image.png

从图中可以看到有四个组件：mongos、config server、shard、replica set。

mongos，数据库集群请求的入口，所有的请求都通过mongos进行协调，不需要在应用程序添加一个路由选择器，mongos自己就是一个请求分发中心，它负责把对应的数据请求请求转发到对应的shard服务器上。在生产环境通常有多mongos作为请求的入口，防止其中一个挂掉所有的mongodb请求都没有办法操作。

config server，顾名思义为配置服务器，存储所有数据库元信息（路由、分片）的配置。mongos本身没有物理存储分片服务器和数据路由信息，只是缓存在内存里，配置服务器则实际存储这些数据。mongos第一次启动或者关掉重启就会从 config server 加载配置信息，以后如果配置服务器信息变化会通知到所有的 mongos 更新自己的状态，这样 mongos 就能继续准确路由。在生产环境通常有多个 config server 配置服务器，因为它存储了分片路由的元数据，防止数据丢失！

shard，分片（sharding）是指将数据库拆分，将其分散在不同的机器上的过程。将数据分散到不同的机器上，不需要功能强大的服务器就可以存储更多的数据和处理更大的负载。基本思想就是将集合切成小块，这些块分散到若干片里，每个片只负责总数据的一部分，最后通过一个均衡器来对各个分片进行均衡（数据迁移）。

replica set，中文翻译副本集，其实就是shard的备份，防止shard挂掉之后数据丢失。复制提供了数据的冗余备份，并在多个服务器上存储数据副本，提高了数据的可用性，并可以保证数据的安全性。

仲裁者（Arbiter），是复制集中的一个MongoDB实例，它并不保存数据。仲裁节点使用最小的资源并且不要求硬件设备，不能将Arbiter部署在同一个数据集节点中，可以部署在其他应用服务器或者监视服务器中，也可部署在单独的虚拟机中。为了确保复制集中有奇数的投票成员（包括primary），需要添加仲裁节点做为投票，否则primary不能运行时不会自动切换primary。

2、整体规划

mongos	port	config	port	shard1	port	shard2	port
192.168.3.18	27017	192.168.3.20(主)	40001	192.168.3.23(主)	10001	192.168.3.25(主)	20001
192.168.3.19	27017	192.168.3.21(副)	40002	192.168.3.24(副)	10002	192.168.3.24(副)	20002
		192.168.3.22(副)	40003	192.168.3.21(仲裁)	10003	192.168.3.22(仲裁)	20003

二、集群部署

1、环境准备

1.1、安装

# wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.6.tgz
# tar -xzvf mongodb-linux-x86_64-3.4.6.tgz
# mv mongodb-linux-x86_64-3.4.6 /data/mongodb
# 创建目录
# mkdir -p /data/app/mongodb/{conf,data,logs}
# mkdir -p /data/app/mongodb/data/相关目录（根据部署情况创建相关目录）
# 例如
# config服务器
# mkdir -p /data/app/mongodb/data/config1
# shard1
# mkdir -p /data/app/mongodb/data/shard1
# shard2
# mkdir -p /data/app/mongodb/data/shard2
# 配置环境变量
# vim /etc/profile
export PATH_HOME=/data/app/mongodb
export PATH=$PATH_HOME/bin:$PATH
# source /etc/profile

1.2、创建认证key

随机一台创建key

# cd /data/app/mongodb/conf
# openssl rand -base64 741 > keyfile
# chmod 600 keyfile
# 发送到其他所有服务器
# 例如
# scp keyfile root@192.168.3.19:/data/app/mongodb/conf/
...........

2、配置config服务器

2.1、配置

2.1.1、192.168.3.20(主)

# vim /data/app/mongodb/conf/config1.conf

# 配置文件内容
logpath = /data/app/mongodb/logs/config1.log
dbpath = /data/app/mongodb/data/config1
configsvr = true
replSet = cfgReplSet
fork = true
port = 40001
logappend = true
logRotate = rename
#bind_ip = 192.168.3.20
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

2.1.2、192.168.3.21(副)

# vim /data/app/mongodb/conf/config1.conf

# 配置文件内容
logpath = /data/app/mongodb/logs/config1.log
dbpath = /data/app/mongodb/data/config1
configsvr = true
replSet = cfgReplSet
fork = true
port = 40002
logappend = true
logRotate = rename
#bind_ip = 192.168.3.21
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

2.1.3、192.168.3.22(副)

# vim /data/app/mongodb/conf/config1.conf

# 配置文件内容
logpath = /data/app/mongodb/logs/config1.log
dbpath = /data/app/mongodb/data/config1
configsvr = true
replSet = cfgReplSet
fork = true
port = 40003
logappend = true
logRotate = rename
#bind_ip = 192.168.3.22
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

2.2、启动

# mongod -f /data/app/mongodb/conf/config1.conf

2.3、初始化配置副本集

# mongo --port 40001
# 新建认证用户
> use admin
> db.createUser(
    {
        user:"admin",
        pwd:"better",
        roles:[{role:"root",db:"admin"}]
    }
)
> db.auth("admin","better")
> rs.initiate({
        "_id":"cfgReplSet",
        "members":[
            {
                "_id":0,
                "host":"192.168.3.20:40001",priority:3
            },
            {
                "_id":1,
                "host":"192.168.3.21:40002",priority:1
            },
            {
                "_id":2,
                "host":"192.168.3.22:40003",priority:1
            }
        ]
   })

cfgReplSet:PRIMARY> rs.status()

3、配置分片副本集

3.1、shard1分片副本集

3.1.1、主节点

# cd /data/app/mongodb/conf
# vim shard1.conf
logpath = /data/app/mongodb/logs/shard1.log
dbpath = /data/app/mongodb/data/shard1/
fork = true
auth = true
port = 10001
shardsvr = true
replSet = shard1
directoryperdb = true
httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
#bind_ip = 192.168.3.23
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

3.1.2、副节点

# cd /data/app/mongodb/conf
# vim shard1.conf
logpath = /data/app/mongodb/logs/shard1.log
dbpath = /data/app/mongodb/data/shard1/
fork = true
auth = true
port = 10002
shardsvr = true
replSet = shard1
directoryperdb = true
httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
#bind_ip = 192.168.3.24
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

3.1.3、仲裁节点

# cd /data/app/mongodb/conf
# vim shard1.conf
logpath = /data/app/mongodb/logs/shard1.log
dbpath = /data/app/mongodb/data/shard1/
fork = true
auth = true
port = 10003
shardsvr = true
replSet = shard1
directoryperdb = true
httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
#bind_ip = 192.168.3.21
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

3.1.4、启动三台机器的shard1副本集

# mongod -f /data/app/mongodb/conf/shard1.conf

3.1.5、初始化副本集

# mongo -port 10001
# 新建认证用户
shard1> use admin
shard1> db.createUser(
    {
        user:"admin",
        pwd:"better",
        roles:[{role:"root",db:"admin"}]
    }
)
shard1> db.auth("admin","better")
shard1> rs.initiate({
        "_id":"shard1",
        "members":[
            {
                "_id":0,
                "host":"192.168.3.23:10001",priority:3
            },
            {
                "_id":1,
                "host":"192.168.3.24:10002",priority:1
            },
            {
                "_id":2,
                "host":"192.168.3.21:10003",arbiterOnly:true
            }
        ]
   })

shard1:PRIMARY> rs.status()

3.2、shard2分片副本集

3.2.1、主节点

# cd /data/app/mongodb/conf
# vim shard2.conf
logpath = /data/app/mongodb/logs/shard2.log
dbpath = /data/app/mongodb/data/shard2/
fork = true
auth = true
port = 20001
shardsvr = true
replSet = shard2
directoryperdb = true
httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
#bind_ip = 192.168.3.25
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

3.2.2、副节点

# cd /data/app/mongodb/conf
# vim shard2.conf
logpath = /data/app/mongodb/logs/shard2.log
dbpath = /data/app/mongodb/data/shard2/
fork = true
auth = true
port = 20002
shardsvr = true
replSet = shard2
directoryperdb = true
httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
#bind_ip = 192.168.3.24
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

3.2.3、仲裁节点

# cd /data/app/mongodb/conf
# vim shard2.conf
logpath = /data/app/mongodb/logs/shard2.log
dbpath = /data/app/mongodb/data/shard2/
fork = true
auth = true
port = 20003
shardsvr = true
replSet = shard2
directoryperdb = true
httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
#bind_ip = 192.168.3.22
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

3.2.4、启动三台机器的shard2副本集

# mongod -f /data/app/mongodb/conf/shard2.conf

3.2.5、初始化副本集

# bin/mongo -port 20001
# 新建认证用户
shard2> use admin
shard2> db.createUser(
    {
        user:"admin",
        pwd:"better",
        roles:[{role:"root",db:"admin"}]
    }
)
shard2> db.auth("admin","better")
shard2> rs.initiate({
        "_id":"shard2",
        "members":[
            {
                "_id":0,
                "host":"192.168.3.25:20001",priority:3
            },
            {
                "_id":1,
                "host":"192.168.3.24:20002",priority:1
            },
            {
                "_id":2,
                "host":"192.168.3.22:20003",arbiterOnly:true
            }
        ]
   })
   
shard2:PRIMARY> rs.status()

4、配置路由服务器 mongos

4.1、192.168.3.18

# cd /data/app/mongodb/conf
# vim mongos.conf
logpath = /data/app/mongodb/logs/mongos.log
configdb=cfgReplSet/192.168.3.20:40001,192.168.3.21:40002,192.168.3.22:40003
fork = true
port = 27017
logappend = true
logRotate = rename
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

4.2、192.168.3.19

# cd /data/app/mongodb/conf
# vim mongos.conf
logpath = /data/app/mongodb/logs/mongos.log
configdb=cfgReplSet/192.168.3.20:40001,192.168.3.21:40002,192.168.3.22:40003
fork = true
port = 27017
logappend = true
logRotate = rename
keyFile = /data/app/mongodb/conf/keyfile
maxConns=20000

4.3、启动

# mongos -f /data/app/mongodb/conf/mongos.conf

4.4、添加认证

mongos> use admin
mongos> db.createUser(
    {
        user:"admin",
        pwd:"better",
        roles:[{role:"root",db:"admin"}]
    }
)
mongos> db.auth("admin","better")
mongos> sh.status()

5、启用分片

随机登录一台mongos

# mongo --port 27017
#使用admin数据库
mongos> user  admin
mongos> db.auth("admin","better")
#串联路由服务器与分配副本集
mongos> sh.addShard("shard1/192.168.3.23:10001,192.168.3.24:10002")

mongos> sh.addShard("shard2/192.168.3.25:20001,192.168.3.24:20002")
#查看集群状态
mongos> sh.status()

三、分片

1、开启库分片

mongos上操作

mongos> use admin
#开启库分片
mongos> sh.enableSharding("库名")                
或
mongos> db.runCommand({enablesharding:"库名"})

2、开启集合分片

#集合分片是基于索引去发片的。
mongos> sh.shardCollection("库.集合",{"name":1})
或
mongos> db.runCommand({shardcollection:"库.集合",key:{"name":1}})
mongos> sh.status()
# 查看集合分布情况
mongos> db.集合.stats()

四、自动化

通过脚本一键部署MongoDB切片集群

#！/bin/bash
#Author: Romi
#Date: 2019-11-15

echo -e "\033[31m 本脚本安装MongoDB-3.40的单机版本或集群。\033[0m"

Single_MongoDB() {
    ssh root@$Sip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    echo "已安装MongoDB"
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/mongodb/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/mongodb.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/mongodb/
fork = true
auth = true
port = 27017
logappend = true
logRotate = rename
#bind_ip = 10.1.0.7
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/mongodb.conf
    sed -i 's/10.1.0.7/$Sip/g' /app_server/mongodb-linux-x86_64-3.4.0/conf/mongodb.conf
    sed -i 's/27017/$Sport/g' /app_server/mongodb-linux-x86_64-3.4.0/conf/mongodb.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/mongodb.conf
eof
}

Cluseter_MongoDB() {
    
echo -e "\033[1;32m 开始初始化MongoDB环境 \033[0m"
echo -ne "参考事例
**********************************************************************************************************************
*请输入config(主、从、从) ip地址和端口(以空格隔开)：192.168.139.16 4001 192.168.139.12 4002 192.168.139.15 4003           *
*选择shard1服务器ip                                                                                                   *
*请输入shard1(主、从、仲裁) ip地址和端口(以空格隔开)：192.168.139.16 10001 192.168.139.12 10002 192.168.139.15 10003      *
*选择shard2服务器ip                                                                                                   *
*请输入shard2(主、从、仲裁) ip地址和端口(以空格隔开)：192.168.139.16 20001 192.168.139.12 20002 192.168.139.15 20003      *
*选择mongos服务器ip                                                                                                   *
*请输入mongos[1..2] ip地址和端口(以空格隔开)：192.168.139.16 2001 192.168.139.12 2002                                   *
*添加认证登录信息                                                                                                      *
*是否添加认证登录信息(Y/y|N/n):y                                                                                       *
*请输入用户名和密码(以空格隔开)：admin 123                                                                              *
**********************************************************************************************************************
"

echo -e "\033[1;34m 选择config服务器ip \033[0m"
#config服务器ip地址和端口
read -p "请输入config(主、从、从) ip地址和端口(以空格隔开)：" configMip configMport configSip configSport configAip configAport

echo -e "\033[1;35m 选择shard1服务器ip \033[0m"
#shard1服务器ip地址和端口
read -p "请输入shard1(主、从、仲裁) ip地址和端口(以空格隔开)：" shard1Mip shard1Mport shard1Sip shard1Sport shard1Aip shard1Aport

echo -e "\033[1;36m 选择shard2服务器ip \033[0m"
#shard2服务器IP地址和端口
read -p "请输入shard2(主、从、仲裁) ip地址和端口(以空格隔开)：" shard2Mip shard2Mport shard2Sip shard2Sport shard2Aip shard2Aport

echo -e "\033[1;33m 选择mongos服务器ip \033[0m"
#mongos服务器IP地址和端口
read -p "请输入mongos[1..2] ip地址和端口(以空格隔开)：" mongosMip mongosMport mongosSip mongosSport
}
Authentication_MongoDB() {
    echo -e "\033[1;32m 添加认证登录信息 \033[0m"
    #使用账号密码进行登录MongoDB集群
    read -p "是否添加认证登录信息(Y/y|N/n):" Auth
    if [ $Auth = 'Y' -o $Auth = 'y' ];then
        read -p "请输入用户名和密码(以空格隔开)：" user secret
    elif [ $Auth = 'N' -o $Auth = 'n' ];then
        read -p "不开启认证，后期可自己添加认证信息！"
    else
        echo "输入错误，请重新输入！"
    fi
}

MongoDB_configM() {
    ssh root@$configMip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/config1/   
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/config1.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/config1
configsvr = true
replSet = cfgReplSet
fork = true
port = $configMport
logappend = true
logRotate = rename
#bind_ip = $configMip
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/config1.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/config1.conf
eof
}
MongoDB_configS() {
    ssh root@$configSip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/config1/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/config1.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/config1
configsvr = true
replSet = cfgReplSet
fork = true
port = $configSport
logappend = true
logRotate = rename
#bind_ip = $configSip
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/config1.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/config1.conf
eof
}
MongoDB_configA() {
    ssh root@$configAip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/config1/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/config1.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/config1
configsvr = true
replSet = cfgReplSet
fork = true
port = $configAport
logappend = true
logRotate = rename
#bind_ip = $configAip
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/config1.conf
echo -ne "//config配置副本集
rs.initiate({
        '_id':'cfgReplSet',
        'members':[
            {
                '_id':0,
                'host':'$configMip:$configMport',priority:3
            },
            {
                '_id':1,
                'host':'$configSip:$configSport',priority:1
            },
            {
                '_id':2,
                'host':'$configAip:$configAport',priority:1
            }
        ]
   })
" > /root/mongorep.js
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/config1.conf
    sleep 2
    bin/mongo 127.0.0.1:$configAport/admin /root/mongorep.js
eof
}

MongoDB_shard1M() {
    ssh root@$shard1Mip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/shard1/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/shard1.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/shard1/
fork = true
port = $shard1Mport
shardsvr = true
replSet = shard1
directoryperdb = true
#httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/shard1.conf
echo -ne "//shard1配置副本集
rs.initiate({
     '_id':'shard1',
        'members':[
            {
                '_id':0,
                'host':'$shard1Mip:$shard1Mport',priority:3
            },
            {
                '_id':1,
                'host':'$shard1Sip:$shard1Sport',priority:1
            },
            {
                '_id':2,
                'host':'$shard1Aip:$shard1Aport',arbiterOnly:true
            }
        ]
   })
" > /root/mongorep.js
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/shard1.conf
    sleep 1
    bin/mongo 127.0.0.1:$shard1Mport/admin /root/mongorep.js
eof
}
MongoDB_shard1S() {
    ssh root@$shard1Sip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/shard1/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/shard1.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/shard1/
fork = true
port = $shard1Sport
shardsvr = true
replSet = shard1
directoryperdb = true
#httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/shard1.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/shard1.conf
eof
}
MongoDB_shard1A() {
    ssh root@$shard1Aip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/shard1/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/shard1.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/shard1/
fork = true
port = $shard1Aport
shardsvr = true
replSet = shard1
directoryperdb = true
#httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/shard1.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/shard1.conf
eof
}

MongoDB_shard2M() {
    ssh root@$shard2Mip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/shard2/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/shard2.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/shard2/
fork = true
port = $shard2Mport
shardsvr = true
replSet = shard2
directoryperdb = true
#httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/shard2.conf
echo -ne "//shard2配置副本集
rs.initiate({
     '_id':'shard2',
        'members':[
            {
                '_id':0,
                'host':'$shard2Mip:$shard2Mport',priority:3
            },
            {
                '_id':1,
                'host':'$shard2Sip:$shard2Sport',priority:1
            },
            {
                '_id':2,
                'host':'$shard2Aip:$shard2Aport',arbiterOnly:true
            }
        ]
   })
" > /root/mongorep.js
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/shard2.conf
    sleep 1
    bin/mongo 127.0.0.1:$shard2Mport/admin /root/mongorep.js
eof
}

MongoDB_shard2S() {
    ssh root@$shard2Sip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/shard2/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/shard2.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/shard2/
fork = true
port = $shard2Sport
shardsvr = true
replSet = shard2
directoryperdb = true
#httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/shard2.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/shard2.conf
eof
}
MongoDB_shard2A() {
    ssh root@$shard2Aip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/data/shard2/
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/shard2.log
dbpath = /app_server/mongodb-linux-x86_64-3.4.0/data/shard2/
fork = true
port = $shard2Aport
shardsvr = true
replSet = shard2
directoryperdb = true
#httpinterface = false
logappend = true
logRotate = rename
oplogSize = 2000
profile = 1
slowms = 500
journal = true
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/shard2.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongod -f conf/shard2.conf
eof
}

MongoDB_mongosM() {
    ssh root@$mongosMip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/mongos.log
configdb=cfgReplSet/$configMip:$configMport,$configSip:$configSport,$configAip:$configAport
fork = true
port = $mongosMport
logappend = true
logRotate = rename
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/mongos.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongos -f conf/mongos.conf
eof
}
MongoDB_mongosS() {
    ssh root@$mongosSip  > /dev/null 2>&1 <<eof
    systemctl stop firewalld
echo -ne "#!/bin/bash
if [ -d /app_server ];then
#    chown -R appuser.appuser /app_server
     echo "/app_server already exists"
else
     mkdir /app_server
fi
if [ -d /app_server/mongodb-linux-x86_64-3.4.0 ];then
    if [ -f /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile ];then
        echo "文件已存在！"
    else
        wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
        chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
    fi
else
    wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz
    tar zxf mongodb-linux-x86_64-3.4.0.tgz
    mv mongodb-linux-x86_64-3.4.0 /app_server/
    mkdir -p /app_server/mongodb-linux-x86_64-3.4.0/{data,conf,logs}
    wget -P /app_server/mongodb-linux-x86_64-3.4.0/conf/ http://47.100.101.150:520/suke/keyfile
    chmod 600 /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
fi
" > MongoDBinstall.sh
    chmod +x MongoDBinstall.sh
    sh MongoDBinstall.sh
echo -ne "#####mongodb3.4配置文件
logpath = /app_server/mongodb-linux-x86_64-3.4.0/logs/mongos.log
configdb=cfgReplSet/$configMip:$configMport,$configSip:$configSport,$configAip:$configAport
fork = true
port = $mongosSport
logappend = true
logRotate = rename
keyFile = /app_server/mongodb-linux-x86_64-3.4.0/conf/keyfile
" > /app_server/mongodb-linux-x86_64-3.4.0/conf/mongos.conf
    cd /app_server/mongodb-linux-x86_64-3.4.0/
    bin/mongos -f conf/mongos.conf
echo -ne "//添加分片
sh.addShard('shard1/$shard1Mip:$shard1Mport,$shard1Sip:$shard1Sport')
sh.addShard('shard2/$shard2Mip:$shard2Mport,$shard2Sip:$shard2Sport')
db.createUser(
    {
        user:'$user',
        pwd:'$secret',
        roles:[{role:'root',db:'admin'}]
    }
)
" > /root/mongorep.js
    sleep 1
    bin/mongo 127.0.0.1:$mongosSport/admin /root/mongorep.js
eof
}


let i=0
while true
do
read -p "请选择你要安装的类型单机(S|s)/集群(C|c):" type
if [ $type = 'S' -o $type = 's' ];then
     read -p "please enter ip and port(Please separate with spaces,0.0.0.0 22):" Sip Sport
     echo "connecting server:$Sip,port is $Sport"
     Single_MongoDB
     break
elif [ $type = 'C' -o $type = 'c' ];then
     Cluseter_MongoDB
     Authentication_MongoDB
     echo -e "\033[1;35m 正在部署config.... \033[0m"
     MongoDB_configM
     MongoDB_configS
     MongoDB_configA
     echo -e "\033[1;36m 正在部署shard1.... \033[0m"
     MongoDB_shard1S
     MongoDB_shard1A
     MongoDB_shard1M
     echo -e "\033[1;33m 正在部署shard2.... \033[0m"
     MongoDB_shard2S
     MongoDB_shard2A
     MongoDB_shard2M
     echo -e "\033[1;34m 正在部署mongos.... \033[0m"
     MongoDB_mongosM
     MongoDB_mongosS
     break
else
     let i+=1
     echo "Enter wrong,please enter again! you have three times, now $i time"
     if [ $i = "3" ];then
     echo "You lost three times！you are Clown! you do not want to install it,so rolling!"
     break
     fi
fi
done