vagrant介绍

vagrant是一个很适合开发者的虚拟环境部署工具，本身集成了主流的虚拟器管理工具，支持vmvare和virtualbox。vagrant的精髓在一个Vagrantfile里面，和docker的Dockerfile功能上一样。我们只需要把需要安装部署的步骤写在Vagrantfile里面，便可以实现轻松部署。vagrant还支持把当前系统做成一个.box后缀命名的镜像，类似docker的image，可轻松实现环境的移植。

集群环境预览

准备

软件

下载并安装如下的软件：

1. virtualbox 官网地址
2. vagrant2.2.7 官网地址
3. kafka_2.11-2.4.0.tgz下载地址
4. 镜像：vagrant-centos-7.2.box
   下载地址：
   链接：https://pan.baidu.com/s/15S2OZq37FcL9RWWSTWntIw
   提取码：3xb0

空间：

15G的硬盘空间

vagrant添加镜像

vagrant box add centos72 D:\\zjdata\\virtualbox\\centos72\\vagrant-centos-7.2.box

vagrant box add命令会在c盘用户下面的.vagrant.d/boxes/目录下面创建一个和镜像大小差不多的文件（700M左右）。

下载vagrant-kafka项目

注意：以下的命令行都是在git-bash下面执行的。

cd /c/zjdata/vagrant-vm/
git clone https://github.com/xueyubingsen/vagrant-kafka.git

初始化vagrant

cd vagrant-kafka
mv Vagrantfile Vagrantfile.bak
vagrant init vagrant-kafka

1. vagrant init 会生成一个Vagrantfile文件和一些隐藏目录，这个vagrant-kafka是随便起的。
2. 先备份一下从github上下载的Vagrantfile文件，等init完之后，再替换回来。

mv Vagrantfile.bak Vagrantfile

修改Vagrantfile

将config.vm.box的值改为我们刚刚添加的box的名字
将KAFKA_VERSION的值改为2.4.0

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  config.vm.box = "centos72"
  config.ssh.forward_agent = true # So that boxes don't have to setup key-less ssh
  config.ssh.insert_key = false # To generate a new ssh key and don't use the default Vagrant one

  vars = {
     "KAFKA_VERSION" => "2.4.0",
     "KAFKA_NAME" => "kafka_2.11-$KAFKA_VERSION",
     "KAFKA_TARGET" => "/vagrant/tars/",
     "KAFKA_HOME" => "$HOME/$KAFKA_NAME"
  }

  # escape environment variables to be loaded to /etc/profile.d/
  as_str = vars.map{|k,str| ["export #{k}=#{str.gsub '$', '\$'}"] }.join("\n")

  # common provisioning for all
  config.vm.provision "shell", path: "scripts/hosts-file-setup.sh", env: vars
  config.vm.provision "shell", inline: "echo \"#{as_str}\" > /etc/profile.d/kafka_vagrant_env.sh", run: "always"
  config.vm.provision "shell", path: "scripts/init.sh", env: vars

  # configure zookeeper cluster
  (1..3).each do |i|
    config.vm.define "zookeeper#{i}" do |s|
      s.vm.hostname = "zookeeper#{i}"
      s.vm.network "private_network", ip: "10.30.3.#{i+1}"
      #s.vm.network "private_network", ip: "10.30.3.#{i+1}", netmask: "255.255.255.0", virtualbox__intnet: "my-network", drop_nat_interface_default_route: true
      s.vm.provision "shell", run: "always", path: "scripts/zookeeper.sh", args:"#{i}", privileged: false, env: vars
    end
  end

  # configure brokers
  (1..3).each do |i|
    config.vm.define "broker#{i}" do |s|
      s.vm.hostname = "broker#{i}"
      s.vm.network "private_network", ip: "10.30.3.#{4-i}0"
      #s.vm.network "private_network", ip: "10.30.3.#{4-i}0", netmask: "255.255.255.0", virtualbox__intnet: "my-network", drop_nat_interface_default_route: true
      s.vm.provision "shell", run: "always", path: "scripts/broker.sh", args:"#{i}", privileged: false, env: vars
    end
  end

  config.vm.provider "virtualbox" do |v|
    #  This setting controls how much cpu time a virtual CPU can use. A value of 50 implies a single virtual CPU can use up to 50% of a single host CPU.
    v.customize ["modifyvm", :id, "--cpuexecutioncap", "50"]
  end
end

拷贝kafka安装包

把开始的时候下载的kafka_2.11-2.4.0.tgz安装包拷贝到项目目录的tars目录下，tars目录需要手工创建。

启动集群

只需要启动集群，vagrant就会按照Vagrantfile里面配置的那样，开始逐个安装集群。

vagrant up

安装之后的样子

$ vagrant global-status
id       name       provider   state   directory
---------------------------------------------------------------------------
436fb44  zookeeper1 virtualbox running C:/zjdata/vagrant-vm/vagrant-kafka
6292373  zookeeper2 virtualbox running C:/zjdata/vagrant-vm/vagrant-kafka
5325c42  zookeeper3 virtualbox running C:/zjdata/vagrant-vm/vagrant-kafka
6b594e2  broker1    virtualbox running C:/zjdata/vagrant-vm/vagrant-kafka
690398d  broker2    virtualbox running C:/zjdata/vagrant-vm/vagrant-kafka
a26ed4b  broker3    virtualbox running C:/zjdata/vagrant-vm/vagrant-kafka

The above shows information about all known Vagrant environments
on this machine. This data is cached and may not be completely
up-to-date (use "vagrant global-status --prune" to prune invalid
entries). To interact with any of the machines, you can go to that
directory and run Vagrant, or you can use the ID directly with
Vagrant commands from any directory. For example:
"vagrant destroy 1a2b3c4d"

注意：可以单独针对某一个节点运行provision

vagrant provision zookeeper1

连接到zookeeper1节点

vagrant ssh zookeeper1