使用 Monit+Mina 监控服务器

服务器不上个监控总是没有安全感,偶尔出点问题,追踪起原因也不是很方便。经过在monitgodeye之间选择,最终选择了monit,最主要的优点就是占用内存很小,而且和ruby无关,所以也不用考虑root用户下ruby环境的问题。
monit的安装和简单使用就不做简单的介绍,大家可以在官网上查看。需要注意的就是建议使用root用户来安装monit,这样就比较方便的使用monit启动其他用户下的服务,不用考虑权限问题。
mina,部署利器,具体使用方法可以到官网查看,不再介绍。
这里主要分享,使用monit+mina对服务器进行监控的一些监控和部署脚本,以及踩过的坑。
部分代码如下:
config/deploy.rb

Dir[File.dirname(__FILE__) + '/mina/*.rb'].each {|file| require file }

environments = {
  'production1' => {
    domain: '115.115.115.115',
    branch: 'master',
    env: 'production1',
    path: "/var/app/wedding",
    user: "root",
    user_home: "/root",
    rvm_home: "/usr/local/rvm",
    thin_ports: 3000..3003
  },
  'production2' => {
    domain: '222.222.222.222',
    branch: 'master',
    env: 'production2',
    path: "/var/app/wedding",
    user: "cloud",
    user_home: "/home/cloud",
    rvm_home: "/usr/local/rvm",
    thin_ports: 2000..2007
  }
}

rails_env = environments.keys.include?(ENV['RAILS_ENV']) ? ENV['RAILS_ENV'] : 'develop'

env_attrs = environments[rails_env]
branch = env_attrs[:branch]
domain = env_attrs[:domain]
rvm_home = env_attrs[:rvm_home]
rvm_path = rvm_home + "/bin/rvm"
env = env_attrs[:env]
path = env_attrs[:path]
user = env_attrs[:user]
rvm = "ruby-2.1.2"
thin_ports = env_attrs[:thin_ports]
user_home = env_attrs[:user_home]

set :rails_env, env
set :domain, domain
set :deploy_to, path
set :repository, 'git@github.com:huoshaoyun/wedding-memo-rails.git'
set :branch, branch
set :ssh_options, '-A'
set :port, '29168' if rails_env == 'production2'
set :user, user
set :term_mode, :nil
set :rvm_path, rvm_path

## script templates
## 根据template文件生成脚本,需要设置必要的变量
set :rvm_home, rvm_home
set :user_home, user_home
set :rvm, rvm
set :path, env_attrs[:path]
set :thin_ports, thin_ports

config/templates/monit/monitrc.erb

#这里是monitrc的配置,省略1000行代码。
include /etc/monit/conf.d/*

config/templates/monit/mysql.erb
记得加上as uid user and gid usergroup,这样就可以以非root用户启动服务。

check process mysqld with pidfile /var/run/mysqld/mysqld.pid
        group mysql
        start program = "/etc/init.d/mysql start" as uid <%= user %> and gid <%= user %>
        stop program = "/etc/init.d/mysql stop" as uid <%= user %> and gid <%= user %>
        if failed host 127.0.0.1 port 3306 then restart
        if 5 restarts within 5 cycles then timeout

config/templates/monit/nginx.erb

check process nginx with pidfile /run/nginx.pid
        start program = "/etc/init.d/nginx start" as uid <%= user %> and gid <%= user %>
        stop  program = "/etc/init.d/nginx stop" as uid <%= user %> and gid <%= user %>

config/templates/monit/redis.erb

# redis
check process redis with pidfile <%= user_home %>/pids/redis.pid
        start = "/bin/bash -c '/bin/sh <%= user_home %>/mysh/startRedis.sh'" as uid <%= user %> and gid <%= user %>
        stop program = "/etc/init.d/redis-server stop" as uid <%= user %> and gid <%= user %>
        group redis

config/templates/monit/sidekiq.erb

check process sidekiq
        with pidfile /var/app/wedding/shared/pids/sidekiq.pid
        start program = "/etc/init.d/sidekiq start" with timeout 90 seconds
        stop program = "/etc/init.d/sidekiq stop" with timeout 90 seconds
        group sidekiq

config/templates/monit/thin.erb

<% thin_ports.each do |thin_port| %>
check process thin-<%= thin_port %> with pidfile /var/app/wedding/tmp/pids/thin.<%= thin_port %>.pid
        start = "/bin/bash -c '/bin/sh <%= user_home %>/mysh/thin.<%= thin_port %>.sh start'" as uid <%= user %> and gid <%= user %>
        stop = "/bin/bash -c '/bin/sh <%= user_home %>/mysh/thin.<%= thin_port %>.sh stop'"  as uid <%= user %> and gid <%= user %>

        if 3 restarts within 5 cycles then timeout
        if failed port <%= thin_port %> protocol http with timeout 30 seconds for 2 cycles then restart
        group thin
<% end %>

config/templates/mysh/redis-server.sh.erb

#!/bin/sh
 
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY 
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT 
# SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED 
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR 
# BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH  
# DAMAGE. 

# 存在bug不能正常start,只能stop
 
### BEGIN INIT INFO
# Provides:             redis-server
# Required-Start:       $syslog
# Required-Stop:        $syslog
# Should-Start:         $local_fs
# Should-Stop:          $local_fs
# Default-Start:        2 3 4 5
# Default-Stop:         0 1 6
# Short-Description:    redis-server - Persistent key-value db
# Description:          redis-server - Persistent key-value db
### END INIT INFO
 
 
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/redis-server
DAEMON_ARGS=/etc/redis.conf
NAME=redis-server
DESC=redis-server
PIDFILE=<%= user_home %>/pids/redis.pid
 
test -x $DAEMON || exit 0
test -x $DAEMONBOOTSTRAP || exit 0
 
set -e
 
case "$1" in
  start)
        echo -n "Starting $DESC: "
        touch $PIDFILE
        chown <%= user %>:<%= user %> $PIDFILE
        if start-stop-daemon --start --quiet --umask 007 --pidfile $PIDFILE --chuid <%= user %>:<%= user %> --exec $DAEMON -- $DAEMON_ARGS
        then
                echo "$NAME."
        else
                echo "failed"
        fi
        ;;
  stop)
        echo -n "Stopping $DESC: "
        if start-stop-daemon --stop --retry 10 --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
        then
                echo "$NAME."
        else
                echo "failed"
        fi
        rm -f $PIDFILE
        ;;
 
  restart|force-reload)
        ${0} stop
        ${0} start
        ;;
  *)
        echo "Usage: /etc/init.d/$NAME {start|stop|restart|force-reload}" >&2
        exit 1
        ;;
esac
 
exit 0

这里存在一个问题不能通过这个脚本来启动redis,只能停止redis,所以又单独写了一个startRedis。
config/templates/mysh/startRedis.sh.erb

#!/bin/sh
cd $HOME && /usr/local/bin/redis-server $HOME/redis-2.8.16/redis.conf 

config/templates/mysh/thin.sh.erb
需要export ruby环境,否则root用户下没有办法来启动thin,而且export PATH="${PATH}:<%= rvm_home %>/gems/<%= rvm %>/bin:/usr/bin:<%= rvm_home %>/bin:<%= rvm_home %>/gems/<%= rvm %>@global/bin:<%= rvm_home %>/rubies/<%= rvm %>/bin/"要将${PATH}放到前面,这样就不会被系统默认的ruby环境覆盖。

#!/bin/bash
 
# Set the environment, as required by Monit
export PATH="${PATH}:<%= rvm_home %>/gems/<%= rvm %>/bin:/usr/bin:<%= rvm_home %>/bin:<%= rvm_home %>/gems/<%= rvm %>@global/bin:<%= rvm_home %>/rubies/<%= rvm %>/bin/"
export GEM_PATH="<%= rvm_home %>/gems/<%= rvm %>:<%= rvm_home %>/gems/<%= rvm %>@global"
export GEM_HOME="<%= rvm_home %>/gems/<%= rvm %>"
 
start () {
  cd <%= path %>/current
  BUNDLE_GEMFILE=<%= path %>/current/Gemfile bundle exec thin -C /etc/thin/wedding.<%= thin_port %>.yml -d start
}
 
stop () {
  kill -s QUIT $(cat <%= deploy_to %>/tmp/pids/thin.<%= thin_port %>.pid)
}
 
case $1 in
  start)
    start
  ;;
  stop)
    stop
  ;;
  *)
  echo $"Usage: $0 {start|stop}"
  exit 1
  ;;
esac
 
exit 0

config/templates/thin.yml.erb

---
chdir: /var/app/wedding/current
environment: <%= rails_env %>
address: 0.0.0.0
port: <%= thin_port %>
timeout: 30
log: /var/app/wedding/shared/log/thin.log
pid: /var/app/wedding/tmp/pids/thin.pid
max_conns: 1024
max_persistent_conns: 100
require: []
wait: 30
servers: <%= servers_number %>
daemonize: true

config/mina/base.rb

require 'erb'

def template(from, to)
  queue %{echo "-----> Put #{from} file to #{to}"}
  erb = File.read(File.expand_path("../../templates/#{from}", __FILE__))
  put ERB.new(erb).result(binding), to
end

def put(content, file)
  queue %[echo #{escape content} > "#{file}"]
end

def escape(str)
  Shellwords.escape(str)
end

config/mina/mysh.rb

namespace :mysh do

  task :setup do
    invoke :'mysh:thin'
    invoke :'mysh:sidekiq'
    invoke :'mysh:redis-server' if rails_env == 'production2'
    %w[startRedis.sh].each do |name|
      destination = "$HOME/mysh/#{name}"
      template "mysh/#{name}.erb", destination
      queue "chmod a+x #{destination}"
    end
  end

  task :thin do
    thin_ports.each do |thin_port|
      set :thin_port, thin_port
      destination = "$HOME/mysh/wedding.#{thin_port}.sh"
      template "mysh/thin.sh.erb", destination
      queue "chmod a+x #{destination}"
    end
  end

  task :sidekiq do
    destination = "$HOME/mysh/sidekiq.sh"
    template "mysh/sidekiq.sh.erb", destination
    queue "chmod a+x #{destination}"
    queue "sudo cp #{destination} /etc/init.d/sidekiq"
  end

  task "redis-server" do
    destination = "$HOME/mysh/redis-server.sh"
    template "mysh/redis-server.sh.erb", destination
    queue "chmod a+x #{destination}"
    queue "sudo cp #{destination} /etc/init.d/redis-server"
  end

end

通过mina mysh:setup来生成服务器相关的启动脚本。

config/mina/thin.rb

namespace :thin do

  task :setup do
    thin_ports.each do |thin_port|
      destination = "/etc/thin/wedding.#{thin_port}.yml"
      set :thin_port, thin_port
      set :servers_number, 1
      template "thin.yml.erb", destination
    end
    invoke :'thin:railsapp'
  end

  task "railsapp" do
    destination = "/etc/thin/railsapp.yml"
    set :thin_port, thin_ports.first
    set :servers_number, thin_ports.size
    template "thin.yml.erb", destination
  end

end

通过mina thin:setup来生成thin的配置文件。
config/mina/monit.rb

namespace :monit do

  desc "Setup all Monit configuration"
  task :setup do
    queue %{echo "-----> Setting up Monit..."}
    monit_config "monitrc", "/etc/monit/monitrc"
    monit_config "monitrc", "/etc/monitrc"
    invoke :'monit:mysql'
    invoke :'monit:redis' if rails_env == 'production2'
    invoke :'monit:nginx'
    invoke :'monit:thin'
    invoke :'monit:sidekiq'
  end
  after "deploy:setup", "monit:setup"

  task(:nginx) { monit_config "nginx" }
  task(:mysql) { monit_config "mysql" }
  task(:redis) { monit_config "redis" }
  task(:thin) { monit_config "thin" }
  task(:sidekiq) { monit_config "sidekiq" }

  %w[start stop syntax reload].each do |command|
    desc "Run Monit #{command} script"
    task command do
      queue %{echo "-----> Monit #{command}"}
      queue "sudo service monit #{command}"
    end
  end

  task :status do
    queue "sudo monit status"
  end

end

def monit_config(name, destination = nil)
  destination ||= "/etc/monit/conf.d/#{name}"
  template "monit/#{name}.erb", "$HOME/monit/#{name}"
  queue "sudo cp $HOME/monit/#{name} #{destination}"
  queue "sudo chown root #{destination}"
  queue "sudo chmod 600 #{destination}"
end

通过mina monit:setup来生成monit的配置文件。 这里需要注意,一定要将monit的一些配置文件的owner修改为root,即和monit安装用户是同一用户,否则是无法启动的。 另外需要注意的是,需要考虑各项服务的pid文件,某些service是默认关闭pid文件的,需要修改相应的配置文件打开。
将配置文件和脚本模版化,可以在配置完一台服务器后,非常迅速的配置完成其他服务器。

一切搞定,这样就可以随时查看服务器的状态,来点效果图。




试着kill掉一个thin


:smiley:

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 196,165评论 5 462
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 82,503评论 2 373
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 143,295评论 0 325
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 52,589评论 1 267
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 61,439评论 5 358
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 46,342评论 1 273
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 36,749评论 3 387
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 35,397评论 0 255
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 39,700评论 1 295
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 34,740评论 2 313
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 36,523评论 1 326
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 32,364评论 3 314
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 37,755评论 3 300
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,024评论 0 19
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 30,297评论 1 251
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 41,721评论 2 342
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 40,918评论 2 336

推荐阅读更多精彩内容