1 实验环境
RabbitMQ 集群
server1.example.com IP: 10.10.10.11 Node: disk
server2.example.com IP: 10.10.10.12 Node: disk
server3.example.com IP: 10.10.10.13 Node: disk
RabbitMQ相关端口
4369 (epmd), 25672 (Erlang distribution)
5672, 5671 (AMQP 0-9-1 without and with TLS)
15672 (if management plugin is enabled)
61613, 61614 (if STOMP is enabled)
1883, 8883 (if MQTT is enabled)
2 YUM方式安装
2.1 添加EPEL源
~># rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
2.2 添加Erlang源
~># yum install wget
~># rpm -Uvh http://packages.erlang-solutions.com/erlang-solutions-1.0-1.noarch.rpm
2.3 安装RabbitMQ
~># wget http://www.rabbitmq.com/releases/rabbitmq-server/v3.6.1/rabbitmq-server-3.6.1-1.noarch.rpm
~># rpm --import https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
~># yum install rabbitmq-server-3.6.1-1.noarch.rpm
3 手动RPM方式
3.1 安装RabbitMQ
~># wget http://www.rabbitmq.com/releases/erlang/erlang-18.3-1.el7.centos.x86_64.rpm
~># wget http://www.rabbitmq.com/releases/rabbitmq-server/v3.6.1/rabbitmq-server-3.6.1-1.noarch.rpm
~># wget https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
~># rpm --import rabbitmq-signing-key-public.asc
~># rpm -Uvh erlang-18.3-1.el7.centos.x86_64.rpm rabbitmq-server-3.6.1-1.noarch.rpm
注: 如果是RedHat6,erlang下载地址为 http://www.rabbitmq.com/releases/erlang/erlang-18.3-1.el6.x86_64.rpm
4 RabbitMQ单机配置
4.1 调整可打开文件/文件描述符数目(高并发支持)
调整系统限制
~># vi /etc/sysctl.conf
fs.file-max = 100000
使设置生效
~># sysctl -p
查看系统限制
~># sysctl fs.file-max
调整用户限制
~># vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
重启系统使之生效,检查用户限制是否生效
~># ulimit -n
4.2 添加/etc/hosts条目
~># echo "192.168.136.201 server1" >> /etc/hosts
4.3 开通防火墙上Web UI访问端口(默认:15672/tcp)
~># firewall-cmd --permanent --add-port=15672/tcp
~># firewall-cmd –-reload
4.4 设置RabbitMQ服务自启动,并启动RabbbitMQ服务
~># chkconfig rabbitmq-server on
~># service rabbitmq-server start
4.5 启用RabbitMQ监控插件
~># rabbitmq-plugins enable rabbitmq_management
4.6 RabbitMQ用户管理
添加用户(用户名root,密码admin)
~># rabbitmqctl add_user admin admin
设置用户角色(设置admin用户为管理员角色)
~># rabbitmqctl set_user_tags admin administrator
设置用户权限(设置admin用户配置、写、读的权限)
~># rabbitmqctl set_permissions -p / admin "." "." ".*"
删除用户(删除guest用户)
~># rabbitmqctl delete_user guest
注意:rabbitmq从3.3.0开始禁止使用guest/guest权限通过除localhost外的访问。
如果想使用guest/guest通过远程机器访问,需要在rabbitmq配置文件中(/etc/rabbitmq/rabbitmq.config)中设置loopback_users为[]。
[{rabbit, [{loopback_users, []}]}].
4.7 登陆Web UI
在浏览器中打开http://10.10.10.11:15672
输入用户名、密码,登陆成功后的界面
4.8 检查RabbitMQ状态
查看进程:
~># ps aux | grep -v grep | grep rabbitmq-server
查看端口:
~># netstat -tnlp | grep 5672
如果提示netstat命令没找到,请先安装net-tools
~># yum install net-tools
查看RabbitMQ状态:
~># rabbitmqctl status
4.9 RabbitMQ默认配置
~># cat /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.1/sbin/rabbitmq-defaults
### next line potentially updated in package install steps
SYS_PREFIX=
### next line will be updated when generating a standalone release
ERL_DIR=
CLEAN_BOOT_FILE=start_clean
SASL_BOOT_FILE=start_sasl
## Set default values
BOOT_MODULE="rabbit"
CONFIG_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq
LOG_BASE=${SYS_PREFIX}/var/log/rabbitmq
MNESIA_BASE=${SYS_PREFIX}/var/lib/rabbitmq/mnesia
ENABLED_PLUGINS_FILE=${SYS_PREFIX}/etc/rabbitmq/enabled_plugins
PLUGINS_DIR="${RABBITMQ_HOME}/plugins"
IO_THREAD_POOL_SIZE=64
CONF_ENV_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq-env.conf
从上面看出:
- 系统prefix是空
- 配置文件路径是 /etc/rabbitmq/rabbitmq.config (erlang会自动加上.config后缀)
- 环境配置文件是 /etc/rabbitmq/rabbitmq-env.conf
- 日志文件目录是 /var/log/rabbitmq
- 插件文件目录是 安装目录下的plugins,这里RPM安装方式下是 /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.1/plugins
5 RabbitMQ集群配置
集群配置在单机配置完成的基础上进行
5.1 添加/etc/hosts条目
在server1、server2、server3里的/etc/hosts文件中分别添加:
10.10.10.11 server1
10.10.10.12 server2
10.10.10.13 server3
5.2 设置每个节点Cookie
Rabbitmq的集群是依赖于erlang的集群来工作的,所以必须先构建起erlang的集群环境。Erlang的集群中各节点是通过一个magic cookie来实现的,这个cookie存放在 /var/lib/rabbitmq/.erlang.cookie 中,文件是400的权限。所以必须保证各节点cookie保持一致,否则节点之间就无法通信
~># chmod 700 /var/lib/rabbitmq/.erlang.cookie
~># echo -n "AZVOCZYZZBVFLBPTBXU" > /var/lib/rabbitmq/.erlang.cookie
~># chmod 400 /var/lib/rabbitmq/.erlang.cookie
建议在RabbitMQ服务启动前修改过cookie,如果RabbitMQ服务已经启动,修改cookie值后,必须重启RabbitMQ服务,这步很关键
~># ps -ef | grep ^rabbitmq | awk '{print $2}' | xargs kill -9
~># service rabbitmq-server start
5.3 开通防火墙上集群通信端口
~># firewall-cmd --permanent --add-port={4369/tcp,25672/tcp}
~># firewall-cmd --reload
5.4 加入集群
将 server1、server2 、server3组成集群:
默认是磁盘节点,如果是内存节点的话,需要加--ram参数
在server2、server3上分别运行:
~># rabbitmqctl stop_app
~># rabbitmqctl join_cluster rabbit@server1
~># rabbitmqctl start_app
5.5 设置镜像策略
~># rabbitmqctl set_policy ha-all "^" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
相关知识
rabbitmqctl (rabbitmq管理工具)
rabbitmq-plugins (rabbitmq插件管理工具)
rabbitmq-server (rabbitmq服务)
主要配置文件
- enabled_plugins:设置允许的插件列表。
- rabbitmq.config:设置rabbitmq运行参数,结构为hash数组格式。
- rabbitmq-env.conf:rabbitmq环境参数配置
rabbitmq-env.conf中的每项都以RABBITMQ_为前缀,常用参数如下:
RABBITMQ_NODENAME=FZTEC-240088 //节点名称
RABBITMQ_NODE_IP_ADDRESS=127.0.0.1 //IP地址,空串bind所有地址,指定地址bind指定网络接口
RABBITMQ_NODE_PORT=5672 //TCP端口号,默认是5672
RABBITMQ_LOG_BASE=/data/rabbitmq/log //日志所在路径
RABBITMQ_PLUGINS_DIR=/data/rabbitmq/plugins //插件所在路径
RABBITMQ_MNESIA_BASE=/data/rabbitmq/mnesia //mnesia所在路径
rabbitmq-env.conf和rabbitmq.config默认是不存在的。
rabbitmq-env.conf需要在缺省位置手动创建一个。
rabbitmq.config需要在RABBITMQ_CONFIG_FILE指定位置手动创建一个。
6 脚本安装RabbitMQ的脚本
创建脚本文件
~># mkdir rabbitmq-script && cd rabbitmq-script
~># vi rabbitmq.config
[
{rabbit,
[
{cluster_partition_handling, pause_minority}
]
}
].
~># vi settings.conf
ERLANG_EL6_RPM=erlang-19.0.4-1.el6.x86_64.rpm
ERLANG_EL6_RPM_URL=http://www.rabbitmq.com/releases/erlang/erlang-19.0.4-1.el6.x86_64.rpm
ERLANG_EL7_RPM=erlang-19.0.4-1.el7.centos.x86_64.rpm
ERLANG_EL7_RPM_URL=http://www.rabbitmq.com/releases/erlang/erlang-19.0.4-1.el7.centos.x86_64.rpm
RABBITMQ_RPM=rabbitmq-server-3.6.9-1.el7.noarch.rpm
RABBITMQ_RPM_URL=http://www.rabbitmq.com/releases/rabbitmq-server/v3.6.9/rabbitmq-server-3.6.9-1.el7.noarch.rpm
RABBITMQ_KEY=rabbitmq-signing-key-public.asc
RABBITMQ_KEY_URL=https://www.rabbitmq.com/rabbitmq-signing-key-public.asc
RABBITMQ_CONFIG_FILE=rabbitmq.config
ERLANG_COOKIE=AZVOCZYZZBVFLBPTBXUD
RABBITMQ_HTTP_PORT=15672
ERL_EPMD_PORT=4369
RABBITMQ_DIST_PORT=25672
ADMIN_PWD=admin
~># vi install_rabbitmq.sh
#!/bin/bash
#====================================================================
# install_rabbitmq.sh
#
# Linux RabbitMQ Auto Install Script
#
# Copyright (c) 2015, Edward Guan <edward.guan@mkcorp.com>
# All rights reserved.
# Distributed under the GNU General Public License, version 3.0.
#
# Intro: http://www.cnblogs.com/edward2013/p/5061511.html
#
#====================================================================
echo "Note: This tiny script has been hardcoded specifically for RHEL/CentOS"
echo ""
if [ $(id -u) != "0" ]; then
echo "Error: You must be root to run this script!"
exit 1
fi
# defind functions
msg() {
printf '%b\n' "$1" >&2
}
success() {
msg "\33[32m[✔]\33[0m ${1}${2}"
}
error() {
msg "\33[31m[✘]\33[0m ${1}${2}"
exit 1
}
program_exists() {
command -v $1 >/dev/null 2>&1
}
function open_rabbitmq_ports() {
if program_exists firewall-cmd; then
firewall-cmd -q --permanent \
--add-port={$RABBITMQ_HTTP_PORT/tcp,$ERL_EPMD_PORT/tcp,$RABBITMQ_DIST_PORT/tcp}
firewall-cmd -q --reload
firewall-cmd -q --query-port $RABBITMQ_HTTP_PORT/tcp || return 1
firewall-cmd -q --query-port $ERL_EPMD_PORT/tcp || return 1
firewall-cmd -q --query-port $RABBITMQ_DIST_PORT/tcp || return 1
fi
}
function erase_rabbitmq() {
rpm -e rabbitmq-server
}
function update_rabbitmq_package() {
local TMP_FILE=/tmp/.$(basename "$0").tmp
# install required package
program_exists wget || yum -y install wget >/dev/null 2>&1
program_exists socat || yum -y install socat >/dev/null 2>&1
# install erlang rpm
if [ ! -f "$ERLANG_RPM_PATH" ]; then
msg "WARNING: $ERLANG_RPM_PATH not found."
msg "Try to download and install from $ERLANG_RPM_URL..."
wget $ERLANG_RPM_URL -P $SCRIPT_PATH || return 1
fi
rpm -U $ERLANG_RPM_PATH 2>$TMP_FILE && success "Installed $ERLANG_RPM_PATH" || {
sed 's/^[ |\t]*//' $TMP_FILE | grep "already installed" || return 1
}
# import rabbitmq key
if [ ! -f "$RABBITMQ_KEY_PATH" ]; then
msg "WARNING: $RABBITMQ_KEY_PATH not found."
msg "Try to download and import from $RABBITMQ_KEY_URL..."
wget $RABBITMQ_KEY_URL -P $SCRIPT_PATH || return 1
fi
rpm --import $RABBITMQ_KEY_PATH && success "Imported $RABBITMQ_KEY_PATH"
# install rabbitmq rpm
if [ ! -f "$RABBITMQ_RPM_PATH" ]; then
msg "WARNING: $RABBITMQ_RPM_PATH not found."
msg "Try to download and install from $RABBITMQ_RPM_URL..."
wget $RABBITMQ_RPM_URL -P $SCRIPT_PATH || return 1
fi
rpm -U $RABBITMQ_RPM_PATH 2>$TMP_FILE && success "Installed $RABBITMQ_RPM_PATH" || {
sed 's/^[ |\t]*//' $TMP_FILE | grep "already installed" || return 1
}
}
function set_rabbitmq_users() {
if program_exists rabbitmqctl; then
rabbitmqctl list_users | grep -q guest && \
rabbitmqctl delete_user guest >/dev/null && \
success "Deleted user [guest]"
rabbitmqctl list_users | grep -q admin || {
rabbitmqctl add_user admin $ADMIN_PWD >/dev/null && \
rabbitmqctl set_user_tags admin administrator >/dev/null && \
rabbitmqctl set_permissions -p / admin ".*" ".*" ".*" >/dev/null && \
success "Added user [admin]"
}
else
msg "Command not find: rabbitmqctl" && return 1
fi
}
function set_rabbitmq_policies() {
if program_exists rabbitmqctl; then
rabbitmqctl list_policies | grep -q ha-all || {
rabbitmqctl set_policy ha-all "^" '{"ha-mode":"all","ha-sync-mode":"automatic"}' >/dev/null && \
success "Added policy [ha-all]"
}
else
msg "Command not find: rabbitmqctl" && return 1
fi
}
function install_rabbitmq() {
rpm -q rabbitmq-server >/dev/null && \
error "$(rpm -q rabbitmq-server) is already installed"
update_rabbitmq_package || return 1
# enable rabbitmq plugin
rabbitmq-plugins enable rabbitmq_management >/dev/null && \
success "Enabled rabbitmq plugin [rabbitmq_management]" || return 1
# open firewall ports
# open_rabbitmq_ports && \
# success "Opened rabbitmq ports [$RABBITMQ_HTTP_PORT/tcp,$ERL_EPMD_PORT/tcp,$RABBITMQ_DIST_PORT/tcp]" || \
# return 1
# deploy rabbitmq.config
RABBITMQ_CONFIG_PATH="$SCRIPT_PATH/$RABBITMQ_CONFIG_FILE"
RABBITMQ_CONFIG_LOCATION=/etc/rabbitmq/rabbitmq.config
cp $RABBITMQ_CONFIG_PATH $RABBITMQ_CONFIG_LOCATION && \
success "Copyed $RABBITMQ_CONFIG_PATH to $RABBITMQ_CONFIG_LOCATION" || return 1
# add host entry into /etc/hosts
HOST_ENTRY=$(egrep "127.0.0.1 +$(hostname -s)" /etc/hosts)
if [ "x$HOST_ENTRY" == "x" ]; then
echo "127.0.0.1 $(hostname -s)" >> /etc/hosts && \
success "Added [127.0.0.1 $(hostname -s)] into /etc/hosts" || return 1
else
msg "[$HOST_ENTRY] is already added into /etc/hosts"
fi
# set erlang cookie
[ "x$COOKIE" == "x" ] && COOKIE="$ERLANG_COOKIE"
ERLANG_COOKIE_PATH=/var/lib/rabbitmq/.erlang.cookie
if [ ! -f "$ERLANG_COOKIE_PATH" ]; then
msg "WARNING: $ERLANG_COOKIE_PATH is not exist, create new one"
touch $ERLANG_COOKIE_PATH || return 1
fi
chmod 700 $ERLANG_COOKIE_PATH
echo -n "$COOKIE" >$ERLANG_COOKIE_PATH
chmod 400 $ERLANG_COOKIE_PATH
chown rabbitmq:rabbitmq $ERLANG_COOKIE_PATH
if [ $(cat $ERLANG_COOKIE_PATH) = "$COOKIE" ]; then
success "Set erlang cookie value to $COOKIE"
fi
}
function join_rabbitmq_cluster() {
local ret=0
if [ "x$SERVER" != "x" ] && [ "$SERVER" != $(hostname -s) ]; then
if program_exists rabbitmqctl; then
rabbitmqctl stop_app >/dev/null
rabbitmqctl reset >/dev/null
if [ "$NODE" == "ram" ]; then
rabbitmqctl join_cluster --ram rabbit@$SERVER >/dev/null || let ret++
else
rabbitmqctl join_cluster rabbit@$SERVER >/dev/null || let ret++
fi
rabbitmqctl start_app >/dev/null
else
msg "Command not find: rabbitmqctl" && return 1
fi
if [ "$ret" -eq 0 ]; then
success "Joined rabbit@$SERVER"
return 0
else
return 1
fi
fi
}
function restart_rabbitmq() {
# kill all rabbitmq server processes
local RABBITMQ_PIDS=$(ps -ef | grep ^rabbitmq | cut -c 9-16 | tr -s "\n" " ")
if [ "x$RABBITMQ_PIDS" != "x" ]; then
kill -9 $RABBITMQ_PIDS && success "Killed all rabbitmq server processes"
fi
# enable and start rabbitmq server
chkconfig rabbitmq-server on
service rabbitmq-server start
}
function print_usage() {
echo "Usage: $(basename "$0") [OPTIONS...]"
echo ""
echo "Options"
echo " [-h|--help] Prints a short help text and exists"
echo " [-i|--install] Install rabbitmq server"
echo " [-u|--update] Update rabbitmq server"
echo " [-e|--erase] Erase (uninstall) rabbitmq server"
echo " [-c|--cookie] <cookie> Set erlang cookie"
echo " [-j|--join] <server> Join rabbitmq cluster"
echo " [-n|--node] <disc|ram> Set cluster node type"
}
# read the options
TEMP=`getopt -o hiuec:j:n: --long help,install,update,erase,cookie:,join:,node: -n $(basename "$0") -- "$@"`
eval set -- "$TEMP"
# extract options and their arguments into variables.
while true; do
case "$1" in
-h|--help) print_usage ; exit 0 ;;
-i|--install) ACTION=install ; shift ;;
-u|--update) ACTION=update ; shift ;;
-e|--erase) ACTION=erase ; shift ;;
-c|--cookie) COOKIE=$2 ; shift 2 ;;
-j|--join) SERVER=$2 ; shift 2 ;;
-n|--node) NODE=$2 ; shift 2 ;;
--) shift ; break ;;
*) error "Internal error!" ;;
esac
done
# get script path
SCRIPT_PATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# load settings
source "$SCRIPT_PATH/settings.conf" || exit 1
# set erlang and rabbitmq rpm & key path
case $(uname -r) in
*el7*)
ERLANG_RPM_PATH="$SCRIPT_PATH/$ERLANG_EL7_RPM";
ERLANG_RPM_URL="$ERLANG_EL7_RPM_URL" ;;
*el6*)
ERLANG_RPM_PATH="$SCRIPT_PATH/$ERLANG_EL6_RPM";
ERLANG_RPM_URL="$ERLANG_EL6_RPM_URL" ;;
*) error "Your system is not RHEL/CentOS" ;;
esac
RABBITMQ_RPM_PATH="$SCRIPT_PATH/$RABBITMQ_RPM"
RABBITMQ_KEY_PATH="$SCRIPT_PATH/$RABBITMQ_KEY"
if [ "x$ACTION" == "x" ] && [ "x$SERVER" == "x" ]; then
error "$(basename "$0"): missing operand\n" \
"Try '$(basename "$0") --help' for more information."
fi
echo "$NODE" | grep -qE "^disk$|^ram$|^$" || {
error "$(basename "$0"): -n: must be disk or ram"
}
# if ACTION is erase, erase rabbitmq server
if [ "$ACTION" == "erase" ]; then
erase_rabbitmq && {
msg "\nThanks for erasing rabbitmq-server."
msg "© `date +%Y` marykay company"
} || error "Failed erase rabbitmq server"
exit 0
fi
# if ACTION is update, update rabbitmq server
if [ "$ACTION" == "update" ]; then
update_rabbitmq_package && restart_rabbitmq && {
msg "\nThanks for updating rabbitmq-server."
msg "© `date +%Y` marykay company"
} || error "Failed update rabbitmq server"
exit 0
fi
# if ACTION is install, install rabbitmq server
if [ "$ACTION" == "install" ]; then
install_rabbitmq && restart_rabbitmq && set_rabbitmq_users && set_rabbitmq_policies && join_rabbitmq_cluster && {
msg "\nThanks for installing rabbitmq-server."
msg "© `date +%Y` marykay company"
} || error "Failed install rabbitmq server"
else
if [ "x$SERVER" != "x" ]; then
join_rabbitmq_cluster && {
msg "\nThanks for joining rabbitmq-server."
msg "© `date +%Y` marykay company"
} || error "Failed join rabbitmq server"
fi
fi
在server1,server2上分别运行脚本
~># chmod +x install_rabbitmq.sh
安装RabbitMQ
~># ./install_rabbitmq.sh -i
升级RabbitMQ
~># ./install_rabbitmq.sh -u
卸载RabbitMQ
~># ./install_rabbitmq.sh -e
加入群集,不带-n参数时,默认节点类型为dsik,-n参数后只能接ram和disk
~># ./install_rabbitmq.sh -j server1 -n ram
安装RabbitMQ并加入群集
~># ./install_rabbitmq.sh -ij server1