1. 前言
目前IT设备对于大多数企业公司来说是必不可少的基础设施，由于云计算的按需付费，便捷配置计算资源（资源包括网络，服务器，存储，应用软件，服务）的特点很受企业的欢迎，因此大多数企业都从传统的IDC迁移到云平台，或者直接到云平台上搭建自己的业务。然而云计算平台很多家，国外有云计算鼻祖亚马AWS，还有微软的Azure，Google的GCP，国内而言比较有名的，阿里巴巴的Alicloud，腾讯的TencentCloud，华为的HuaweiCloud。
在运维领域中，针对这些云平台的IT资源管理也是重中之重。尤其是最近几逐渐流行的DevOps理念，Infrastructure as Code（基础架构即代码）文化，所以在社区的贡献下，Terraform这个工具应用而生，Terraform是来自HashiCorp家族，因此采用了 HashiCorp 配置语言 (HCL)，Terraform的意义在于，通过同一套规则和命令来操作不同的云平台（包括私有云）。详情可参考官方文档https://www.terraform.io。
工作原因接触Terraform一年多，认为它是自动化配置与编排必备利器，记录下自己使用这个工具在AWS上的一些实践。本次文章不会过多的讲解基础概念，而是注重实践，参考AWS VPC的官方文档中场景2：搭建带有公有子网和私有子网 (NAT) 的 VPC https://docs.aws.amazon.com/zh_cn/vpc/latest/userguide/VPC_Scenario2.html
假如没有Terraform这个工具，我们会通过AWS的控制台或者命令行去依次建立VPC，子网，路由表，关联路由表等等，如果业务拓展，还得在重复在去创建，而且这些创建的资源都是无状态的。这种网络资源应该是不可变，如果有任何修改，故障，我们都不好排查故障。有了Terraform以后就好办了，基础设施代码化。Terraform里面核心有两类文件，编排配置文件和状态文件。编排配置文件是编排的资源，比如是VPC，Subnet等，状态文件是记录当前资源的状态。
2. 安装Terraform
以下是官方安装教程https://www.terraform.io/intro/getting-started/install.html，下载解压，配置环境变量即可。以下是我的环境
[centos@ip-10-20-6-165 ~]$ uname -r
3.10.0-514.26.2.el7.x86_64
[centos@ip-10-20-6-165 ~]$ cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)
[centos@ip-10-20-6-165 ~]$ ll /opt/terraform*
-rwxrwxr-x 1 centos centos 69122624 Apr 10 2018 /opt/terraform
-rw-r--r-- 1 root root 16490308 Apr 10 2018 /opt/terraform_0.11.7_linux_amd64.zip
[centos@ip-10-20-6-165 ~]$ grep opt /etc/profile
export PATH=$PATH:/opt
[centos@ip-10-20-6-165 ~]$ which terraform
/opt/terraform
[centos@ip-10-20-6-165 ~]$ terraform --version
Terraform v0.11.7
Your version of Terraform is out of date! The latest version
is 0.11.9. You can update by downloading from www.terraform.io/downloads.html
3. 编写配置
3.1 目录配置路径规划，VPC规划
下面是我已经编辑好目录架构和配置，仅供参考，可以通过代码管理工具提交仓库供多人协作。每个目录的作用我已经详细在后面添加注释，现在只是建立stg的vpc的网络架构，只在目录rubin/terraform/vpc/stg_cn_rubin下操作，其他空的目录是资源还没有建立，以后会逐步添加，比如rubin/terraform/iam目录，以后肯定会创建stg环境的和prd环境的，其他rubin/terraform/s3，rubin/terraform/shard目录也是如此。
[centos@ip-10-20-6-165 ~]$ cd rubin/
[centos@ip-10-20-6-165 rubin]$
[centos@ip-10-20-6-165 rubin]$ tree
.
└── terraform #Terraform主目录，存放项目所有Terraform相关的代码
├── iam #权限管理相关资源的代码
├── modules #可以公用的资源代码
├── s3 #创建S3资源的代码
├── shard #创建每个服务相关EC2，ALB，SG的代码
└── vpc #创建网络架构代码
├── prd_cn_rubin #prd环境的网络架构代码
└── stg_cn_rubin #stg环境的网络架构代码
├── backend.tf
├── bastion.tf
├── common.tf
├── outputs.tf
├── terraform.tfvars
├── variables.tf
└── vpc.tf
8 directories, 7 files
在建立VPC之前应该根据业务情况划分好子网，比如提前规划选好CIDR，如果将来业务有三方合作需要打通网络，或者需要与自有公司传统的IDC网络打通，就要注意这一方面，下面是我规划的CIDR
VPC.PNG
3.2 配置文件详解
就经验而言，我编写了7个配置文件，下面我详细说明每个文件的作用，仅供参考。
[centos@ip-10-20-6-165 rubin]$ pwd
/home/centos/rubin
[centos@ip-10-20-6-165 rubin]$ cd terraform/vpc/
prd_cn_rubin/ stg_cn_rubin/
[centos@ip-10-20-6-165 rubin]$ cd terraform/vpc/stg_cn_rubin/
[centos@ip-10-20-6-165 stg_cn_rubin]$ pwd
/home/centos/rubin/terraform/vpc/stg_cn_rubin
[centos@ip-10-20-6-165 stg_cn_rubin]$ ll
total 32
-rw-rw-r-- 1 centos centos 285 Sep 21 07:24 backend.tf
-rw-rw-r-- 1 centos centos 1723 Sep 25 07:54 bastion.tf
-rw-rw-r-- 1 centos centos 977 Sep 25 08:10 common.tf
-rw-rw-r-- 1 centos centos 593 Sep 25 09:27 outputs.tf
-rw-rw-r-- 1 centos centos 173 Sep 21 07:24 terraform.tfvars
-rw-rw-r-- 1 centos centos 901 Sep 25 09:48 variables.tf
-rw-rw-r-- 1 centos centos 6676 Oct 20 03:09 vpc.tf
3.2.1 backend.tf
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat backend.tf
provider "aws" {
region = "${var.aws_region}" # 引用了变量，变量值在variables.tf中
}
terraform {
required_version = ">= 0.11.7"
backend "s3" {
encrypt = "true"
bucket = "rubin-cn-stg-terraform-state"
region = "cn-north-1"
key = "vpc/stg_cn_rubin/terraform.tfstate"
dynamodb_table = "terraform-lock"
}
}
这个文件是定义provider和远程存储terraform.tfstate的s3存储桶，provider是Terraform定制的一套接口，阿里云、AWS、私有云等如果想接入进来被Terraform编排和管理就要实现一套Provider，官网https://www.terraform.io/docs/providers/index.html，我使用的AWS的云平台，所以使用的AWS的provider。
关于terraform.tfstate，我在前言中说过，状态文件是记录当前资源的状态。每次运行terraform apply时，都会把最新的配置和当前状态文件中的内容进行比较后，再做更改。所以这个文件不能乱改，团队协作时可以加个锁，因此手动通过AWS console添加了一个存储桶，然后再加了一个dynamodb_table的锁。
image.png
image.png
3.2.2 variables.tf
这个文件相当于申明了variable，可以在此加一个默认的值
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat variables.tf
variable "aws_region" {
default = "cn-north-1"
}
variable "vpc_name" {
description = "The name of the VPC"
}
variable "cidr_numeral" {
description = "The VPC CIDR numeral (10.x.0.0/16)"
}
variable "cidr_numeral_public" {
default = {
"0" = "0"
"1" = "1"
"2" = "2"
}
}
variable "cidr_numeral_private" {
default = {
"0" = "3"
"1" = "4"
"2" = "5"
}
}
variable "cidr_numeral_private_db" {
default = {
"0" = "6"
"1" = "7"
"2" = "8"
}
}
variable "cidr_numeral_private_emr" {
default = {
"0" = "9"
"1" = "10"
"2" = "11"
}
}
variable "ssh_key_name" {
description = "A master ssh key"
}
variable "bastion_image" {
default = {
cn-north-1 = "ami-7866b115"
}
}
variable "env" {
description = "The AWS env tag"
}
variable "availability_zones" {
description = "A comma-delimited list of availability zones for the VPC."
}
3.2.3 terraform.tfvars
此文件是在文件申明的变量赋值，赋值可以供当前目录的其他的文件引用
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat terraform.tfvars
aws_region = "cn-north-1"
cidr_numeral = "101"
vpc_name = "rubin_stg_cn"
ssh_key_name = "rubin-stg-cn-master"
env = "staging"
availability_zones = "cn-north-1a,cn-north-1b"
3.2.4 vpc.tf
这个文件就是主文件，用来建立VPC，subnet，gateway，natgateway，路由表，路由表关联等一系列操作。其中有些是Terraform的内部函数，关于内部函数可参考https://www.terraform.io/docs/configuration/interpolation.html
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat vpc.tf
# VPC DESIGN
resource "aws_vpc" "default" {
cidr_block = "10.${var.cidr_numeral}.0.0/16"
enable_dns_hostnames = true
tags {
Name = "vpc-${var.vpc_name}"
}
}
resource "aws_internet_gateway" "default" {
vpc_id = "${aws_vpc.default.id}"
tags {
Name = "igw-${var.vpc_name}"
}
}
resource "aws_eip" "nat" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc = true
tags {
Name = "ip-NAT-${var.vpc_name}"
}
}
resource "aws_nat_gateway" "nat" {
count = "${length(split(",", "${var.availability_zones}"))}"
allocation_id = "${element(aws_eip.nat.*.id, count.index)}"
subnet_id = "${element(aws_subnet.public.*.id, count.index)}"
tags {
Name = "gw-NAT-${var.vpc_name}"
}
}
# PUBLIC SUBNETS
# The public subnet is where the bastion, NATs and ELBs reside. In most cases,
# there should not be any servers in the public subnet.
resource "aws_subnet" "public" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc_id = "${aws_vpc.default.id}"
cidr_block = "10.${var.cidr_numeral}.${lookup(var.cidr_numeral_public, count.index)}.0/24"
availability_zone = "${element(split(",", var.availability_zones), count.index)}"
map_public_ip_on_launch = true
tags {
Name = "public${count.index}-${var.vpc_name}"
immutable_metadata = "{ \"purpose\": \"external_${var.vpc_name}\", \"target\": null }"
}
}
# PUBLIC SUBNETS - Default route
#
resource "aws_route_table" "public" {
vpc_id = "${aws_vpc.default.id}"
route {
cidr_block = "0.0.0.0/0"
gateway_id = "${aws_internet_gateway.default.id}"
}
tags {
Name = "publicrt-${var.vpc_name}"
}
}
# PUBLIC SUBNETS - Route associations
#
resource "aws_route_table_association" "public" {
count = "${length(split(",", "${var.availability_zones}"))}"
subnet_id = "${element(aws_subnet.public.*.id, count.index)}"
route_table_id = "${aws_route_table.public.id}"
}
# PRIVATE SUBNETS
#
# Route Tables in a private subnet will not have Route resources created
# statically for them as the NAT instances are responsible for dynamically
# managing them on a per-AZ level using the Network=Private tag.
resource "aws_subnet" "private" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc_id = "${aws_vpc.default.id}"
cidr_block = "10.${var.cidr_numeral}.${lookup(var.cidr_numeral_private, count.index)}.0/24"
availability_zone = "${element(split(",", var.availability_zones), count.index)}"
tags {
Name = "private${count.index}-${var.vpc_name}"
immutable_metadata = "{ \"purpose\": \"internal_${var.vpc_name}\", \"target\": null }"
Network = "Private"
}
}
resource "aws_route_table" "private" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc_id = "${aws_vpc.default.id}"
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = "${element(aws_nat_gateway.nat.*.id, count.index)}"
}
tags {
Name = "private${count.index}rt-${var.vpc_name}"
Network = "Private"
}
}
resource "aws_route_table_association" "private" {
count = "${length(split(",", "${var.availability_zones}"))}"
subnet_id = "${element(aws_subnet.private.*.id, count.index)}"
route_table_id = "${element(aws_route_table.private.*.id, count.index)}"
}
# PRIVATE SUBNETS (DB)
#
# Route Tables in a private subnet will not have Route resources created
# statically for them as the NAT instances are responsible for dynamically
# managing them on a per-AZ level using the Network=Private tag.
resource "aws_subnet" "private_db" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc_id = "${aws_vpc.default.id}"
cidr_block = "10.${var.cidr_numeral}.${lookup(var.cidr_numeral_private_db, count.index)}.0/24"
availability_zone = "${element(split(",", var.availability_zones), count.index)}"
tags {
Name = "db-private${count.index}-${var.vpc_name}"
immutable_metadata = "{ \"purpose\": \"internal_${var.vpc_name}\", \"target\": null }"
Network = "Private"
}
}
resource "aws_route_table" "private_db" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc_id = "${aws_vpc.default.id}"
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = "${element(aws_nat_gateway.nat.*.id, count.index)}"
}
tags {
Name = "privatedb${count.index}rt-${var.vpc_name}"
Network = "Private"
}
}
resource "aws_route_table_association" "private_db" {
count = "${length(split(",", "${var.availability_zones}"))}"
subnet_id = "${element(aws_subnet.private_db.*.id, count.index)}"
route_table_id = "${element(aws_route_table.private_db.*.id, count.index)}"
}
# PRIVATE SUBNETS
#
# Route Tables in a private subnet will not have Route resources created
# statically for them as the NAT instances are responsible for dynamically
# managing them on a per-AZ level using the Network=Private tag.
resource "aws_subnet" "private_emr" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc_id = "${aws_vpc.default.id}"
cidr_block = "10.${var.cidr_numeral}.${lookup(var.cidr_numeral_private_emr, count.index)}.0/24"
availability_zone = "${element(split(",", var.availability_zones), count.index)}"
tags {
Name = "private_emr${count.index}-${var.vpc_name}"
immutable_metadata = "{ \"purpose\": \"internal_${var.vpc_name}\", \"target\": null }"
Network = "Private"
}
}
resource "aws_route_table" "private_emr" {
count = "${length(split(",", "${var.availability_zones}"))}"
vpc_id = "${aws_vpc.default.id}"
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = "${element(aws_nat_gateway.nat.*.id, count.index)}"
}
tags {
Name = "private_emr${count.index}rt-${var.vpc_name}"
Network = "Private"
}
}
resource "aws_route_table_association" "private_emr" {
count = "${length(split(",", "${var.availability_zones}"))}"
subnet_id = "${element(aws_subnet.private_emr.*.id, count.index)}"
route_table_id = "${element(aws_route_table.private_emr.*.id, count.index)}"
}
3.2.5 outputs.tf
当我们创建的资源后，经常需要知道这些资源的ID，因此定义一个output，将我们想要的资源ID显示出来或者输出到文件，从而避免我们在去控制台上查询获取这些信息。Terraform的出参就像是存过的产出，开发人员可以在编排时定义output出参来指定自己关心的内容，该内容会在任务执行的日志中高亮显示，而且在任务执行完毕后我们可以通过terrafomr output var_name的方式查看参数结果。
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat outputs.tf
output "vpc_id" {
value = "${aws_vpc.default.id}"
}
output "cidr_block" {
value = "${aws_vpc.default.cidr_block}"
}
output "private_subnets" {
value = "${join(",", aws_subnet.private.*.id)}"
}
output "public_subnets" {
value = "${join(",", aws_subnet.public.*.id)}"
}
output "private_db_subnets" {
value = "${join(",", aws_subnet.private_db.*.id)}"
}
output "private_emr_subnets" {
value = "${join(",", aws_subnet.private_emr.*.id)}"
}
output "nat_eip" {
value = "${join(",", aws_eip.nat.*.public_ip)}"
}
output "bastion_eip" {
value = "${aws_eip.bastion.public_ip}"
}
3.2.6 bastion.tf
当VPC建立好的时候，我们只需要通过一台有公网IP的跳板机，去访问私网的机器，这个文件就是创建一个跳板机的安全组，能通过IP(1...*/28 )访问跳板机的22，2022端口，80，443，53，123端口流量能出，通过安全组限制网络进出流量。安全起见，我对其中IP加密显示，可以自行更改。更严格的还可以通过VPC配置的Network ACLs去控制。
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat bastion.tf
resource "aws_security_group" "bastion" {
name = "bastion-${var.vpc_name}"
description = "Allows SSH access to the bastion server"
vpc_id = "${aws_vpc.default.id}"
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["1.*.*.*/28"]
description = "Office IP"
}
ingress {
from_port = 2022
to_port = 2022
protocol = "tcp"
cidr_blocks = ["1.*.*.*/28"]
description = "Office IP"
}
egress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 53
to_port = 53
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 53
to_port = 53
protocol = "udp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 123
to_port = 123
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 123
to_port = 123
protocol = "udp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["10.0.0.0/8"]
}
tags {
Name = "bastion-${var.vpc_name}"
}
}
# EIP for bastion
resource "aws_eip" "bastion" {
vpc = true
}
3.2.7 common.tf
这个文件是创建了一个共有的安全组给私网地址的机器使用。能接受当前VPC下私网10.0.0.0/8网段的任何流量，流量能从80，443，53，123的端口出。
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat common.tf
resource "aws_security_group" "internal_common" {
name = "internal_common-${var.vpc_name}"
description = "commonly used security group for ocf instances"
vpc_id = "${aws_vpc.default.id}"
ingress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["10.0.0.0/8"]
}
egress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress { # dns
from_port = 53
to_port = 53
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress { # dns
from_port = 53
to_port = 53
protocol = "udp"
cidr_blocks = ["0.0.0.0/0"]
}
egress { # ntp
from_port = 123
to_port = 123
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress { # ntp
from_port = 123
to_port = 123
protocol = "udp"
cidr_blocks = ["0.0.0.0/0"]
}
}
4. 创建资源
在步骤三中，创建的资源的code都已经写好，接下来就可以通过AWS 的access key去执行Terraform 命令创建资源。
4.1 AWS console生成Access key并配置
创建Access key 参考官网https://docs.aws.amazon.com/zh_cn/IAM/latest/UserGuide/id_credentials_access-keys.html
配置Access key 参考官网https://docs.aws.amazon.com/zh_cn/cli/latest/userguide/cli-chap-getting-started.html
[centos@ip-10-20-6-165 ~]$ aws configure --profile rubin-stg
AWS Access Key ID [****************M67J]:
AWS Secret Access Key [****************mgNp]:
Default region name [cn-north-1]:
Default output format [json]:
[centos@ip-10-20-6-165 ~]$ sed -n '24,26p' /home/centos/.aws/config
[profile rubin-stg]
output = json
region = cn-north-1
[centos@ip-10-20-6-165 ~]$ sed -n '25,27p' /home/centos/.aws/credentials
[rubin-stg]
aws_access_key_id = ****************M67J
aws_secret_access_key = ****************mgNp
4.2 Terrform get & init
执行terraform init命令，就像git init一样，对当前目录做初始化，下载tf中的provider，并为后续的操作准备必要的环境条件
[centos@ip-10-20-6-165 ~]$ cd rubin/terraform/vpc/stg_cn_rubin/
[centos@ip-10-20-6-165 stg_cn_rubin]$ pwd
/home/centos/rubin/terraform/vpc/stg_cn_rubin
[centos@ip-10-20-6-165 stg_cn_rubin]$ export AWS_DEFAULT_PROFILE=rubin-stg && export AWS_PROFILE=rubin-stg
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform --help
Usage: terraform [--version] [--help] <command> [args]
The available commands for execution are listed below.
The most common, useful commands are shown first, followed by
less common or more advanced commands. If you're just getting
started with Terraform, stick with the common commands. For the
other commands, please read the help and docs before usage.
Common commands:
apply Builds or changes infrastructure
console Interactive console for Terraform interpolations
destroy Destroy Terraform-managed infrastructure
env Workspace management
fmt Rewrites config files to canonical format
get Download and install modules for the configuration
graph Create a visual graph of Terraform resources
import Import existing infrastructure into Terraform
init Initialize a Terraform working directory
output Read an output from a state file
plan Generate and show an execution plan
providers Prints a tree of the providers used in the configuration
push Upload this Terraform module to Atlas to run
refresh Update local state file against real resources
show Inspect Terraform state or plan
taint Manually mark a resource for recreation
untaint Manually unmark a resource as tainted
validate Validates the Terraform files
version Prints the Terraform version
workspace Workspace management
All other commands:
debug Debug output management (experimental)
force-unlock Manually unlock the terraform state
state Advanced state management
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform get #编写配置文件中没有导入module，所以执行没有结果，如果配置文件有导入module应先执行
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform init
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (1.41.0)...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.aws: version = "~> 1.41"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
4.3 Terraform plan
预览执行计划，不是必须，可以预览要创建的资源,终端日志太长，只粘贴一部分
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_eip.bastion
id: <computed>
allocation_id: <computed>
association_id: <computed>
domain: <computed>
instance: <computed>
network_interface: <computed>
private_ip: <computed>
public_ip: <computed>
vpc: "true"
+ aws_eip.nat[0]
id: <computed>
allocation_id: <computed>
association_id: <computed>
domain: <computed>
instance: <computed>
network_interface: <computed>
private_ip: <computed>
public_ip: <computed>
tags.%: "1"
tags.Name: "ip-NAT-rubin_stg_cn"
vpc: "true"
+ aws_eip.nat[1]
id: <computed>
allocation_id: <computed>
association_id: <computed>
domain: <computed>
...
+ aws_subnet.public[1]
id: <computed>
arn: <computed>
assign_ipv6_address_on_creation: "false"
availability_zone: "cn-north-1b"
cidr_block: "10.101.1.0/24"
ipv6_cidr_block: <computed>
ipv6_cidr_block_association_id: <computed>
map_public_ip_on_launch: "true"
tags.%: "2"
tags.Name: "public1-rubin_stg_cn"
tags.immutable_metadata: "{ \"purpose\": \"external_rubin_stg_cn\", \"target\": null }"
vpc_id: "${aws_vpc.default.id}"
+ aws_vpc.default
id: <computed>
arn: <computed>
assign_generated_ipv6_cidr_block: "false"
cidr_block: "10.101.0.0/16"
default_network_acl_id: <computed>
default_route_table_id: <computed>
default_security_group_id: <computed>
dhcp_options_id: <computed>
enable_classiclink: <computed>
enable_classiclink_dns_support: <computed>
enable_dns_hostnames: "true"
enable_dns_support: "true"
instance_tenancy: "default"
ipv6_association_id: <computed>
ipv6_cidr_block: <computed>
main_route_table_id: <computed>
tags.%: "1"
tags.Name: "vpc-rubin_stg_cn"
Plan: 32 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
4.4 Terraform apply
真正执行编排计划,创建资源的终端日志太长,只粘贴一部分，几分钟后整个资源全被创建完成，最后output把当前资源的ID显示出来了
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_eip.bastion
id: <computed>
allocation_id: <computed>
association_id: <computed>
...
tags.Name: "vpc-rubin_stg_cn"
Plan: 32 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_vpc.default: Creating...
arn: "" => "<computed>"
assign_generated_ipv6_cidr_block: "" => "false"
cidr_block: "" => "10.101.0.0/16"
default_network_acl_id: "" => "<computed>"
default_route_table_id: "" => "<computed>"
...
aws_route_table_association.private[0]: Creation complete after 0s (ID: rtbassoc-0dec0c6caa85081aa)
aws_route_table_association.private[1]: Creation complete after 0s (ID: rtbassoc-00541f1193c1f006a)
aws_route_table_association.private_db[0]: Creation complete after 0s (ID: rtbassoc-0707dc991b6f0b502)
aws_route_table_association.private_db[1]: Creation complete after 0s (ID: rtbassoc-0e1c00b46b31e13a6)
aws_route_table_association.private_emr[1]: Creation complete after 0s (ID: rtbassoc-0dcfc3ec369382fa4)
aws_route_table_association.private_emr[0]: Creation complete after 0s (ID: rtbassoc-0ed91fcc9955b3674)
Apply complete! Resources: 32 added, 0 changed, 0 destroyed.
Outputs:
bastion_eip = 54.223.216.43
cidr_block = 10.101.0.0/16
nat_eip = 54.222.176.3,54.222.249.234
private_db_subnets = subnet-095f9065be2e9bb6d,subnet-0d9fdec0b255f17c1
private_emr_subnets = subnet-090653a4e725b8d90,subnet-0d0fd140f10721f60
private_subnets = subnet-0f34c39ec475c0384,subnet-095341650c1b43695
public_subnets = subnet-06bb6bea3d757253b,subnet-03307ab8f793fd944
vpc_id = vpc-02d9520415468c7f0
5. 检查
5.1 资源检查
现在整个VPC资源已经完全建立好，可以通过AWS console查看
VPC
image.png
Subnets
image.png
Route tables
image.png
Internet Gateway
image.png
Elastic IPs
image.png
NAT Gateways
image.png
5.2 网络检查
然后启动一个带公网的跳板机实例和一个在仅有私网地址的实例，也是通过Terrform去建立，然后把上文创建的一个eip通过控制台手动绑定bastion的跳板机，连接登录私网实例，测试网络连接，代码如下
[centos@ip-10-20-6-165 stg_cn_rubin]$ ll
total 36
-rw-rw-r-- 1 centos centos 285 Sep 21 07:24 backend.tf
-rw-rw-r-- 1 centos centos 1723 Sep 25 07:54 bastion.tf
-rw-rw-r-- 1 centos centos 977 Sep 25 08:10 common.tf
-rw-rw-r-- 1 centos centos 791 Oct 22 10:47 network_test.tf
-rw-rw-r-- 1 centos centos 593 Sep 25 09:27 outputs.tf
-rw-rw-r-- 1 centos centos 173 Sep 21 07:24 terraform.tfvars
-rw-rw-r-- 1 centos centos 901 Sep 25 09:48 variables.tf
-rw-rw-r-- 1 centos centos 6676 Oct 20 03:09 vpc.tf
[centos@ip-10-20-6-165 stg_cn_rubin]$ cat network_test.tf
resource "aws_instance" "rubin-bastion" {
ami = "ami-9a06def7"
availability_zone = "cn-north-1a"
instance_type = "t2.small"
key_name = "${var.ssh_key_name}"
subnet_id = "subnet-06bb6bea3d757253b"
security_groups = ["${aws_security_group.bastion.id}"]
tags {
Name = "rubin-bastion"
terraform = "true"
}
}
resource "aws_instance" "rubin-network-test" {
ami = "ami-9a06def7"
availability_zone = "cn-north-1a"
instance_type = "t2.small"
key_name = "${var.ssh_key_name}"
subnet_id = "subnet-0f34c39ec475c0384"
security_groups = ["${aws_security_group.internal_common.id}"]
tags {
Name = "network-test-private"
terraform = "true"
}
}
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform get
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform init
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform plan
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform apply
[centos@ip-10-20-6-165 stg_cn_rubin]$ terraform apply
aws_eip.bastion: Refreshing state... (ID: eipalloc-026fd9c76126d82cf)
aws_eip.nat[0]: Refreshing state... (ID: eipalloc-0453f9f654f67ed43)
aws_vpc.default: Refreshing state... (ID: vpc-02d9520415468c7f0)
aws_eip.nat[1]: Refreshing state... (ID: eipalloc-0a7e8939e5dfdaeec)
aws_subnet.private_db[0]: Refreshing state... (ID: subnet-095f9065be2e9bb6d)
......
aws_instance.rubin-network-test: Still creating... (10s elapsed)
aws_instance.rubin-bastion: Still creating... (10s elapsed)
aws_instance.rubin-bastion: Still creating... (20s elapsed)
aws_instance.rubin-network-test: Still creating... (20s elapsed)
aws_instance.rubin-network-test: Creation complete after 22s (ID: i-0808e36648559a041)
aws_instance.rubin-bastion: Still creating... (30s elapsed)
aws_instance.rubin-bastion: Creation complete after 32s (ID: i-03aafbf3af96821e5)
然后通过AWS console查看启动的实例，IP是随机生成的，我们还得把上一步的EIP绑定到当前跳板机的实例。
image.png
绑定EIP后如图下图，bastion跳板机实例的Public IP已经变化，私网实例由于在私网中，并没有公网IP，需要通过bastion跳板连接
image.png
然后通过提前在aws 上建立好的key去连接bastion跳板机实例，由于测试，使用的AMI是AWS上社区的，使用默认的ec2-user
[centos@ip-10-20-6-165 stg_cn_rubin]$ ll /home/centos/rubin-stg-cn-master.pem
-rw------- 1 centos centos 1692 Sep 21 07:06 /home/centos/rubin-stg-cn-master.pem
[ec2-user@ip-10-101-0-15 ~]$ curl ifconfig.me #查看当前实例IP
54.223.216.43
[ec2-user@ip-10-101-0-15 ~]$ curl -I www.baidu.com #测试网络
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
Connection: Keep-Alive
Content-Length: 277
Content-Type: text/html
Date: Mon, 22 Oct 2018 13:46:03 GMT
Etag: "575e1f60-115"
Last-Modified: Mon, 13 Jun 2016 02:50:08 GMT
Pragma: no-cache
Server: bfe/1.0.8.18
[ec2-user@ip-10-101-0-15 ~]$ nslookup www.baidu.com
Server: 10.101.0.2
Address: 10.101.0.2#53
Non-authoritative answer:
www.baidu.com canonical name = www.a.shifen.com.
Name: www.a.shifen.com
Address: 220.181.111.188
Name: www.a.shifen.com
Address: 220.181.112.244
以上bastion跳板机具有公网IP能够上网，接下来连接私网实例测试，首先通过控制台查找私网IP
image.png
[ec2-user@ip-10-101-0-15 ~]$ ll
total 4
-rw------- 1 ec2-user ec2-user 1671 Oct 22 13:55 rubin-stg-cn-master.pem
[ec2-user@ip-10-101-0-15 ~]$
[ec2-user@ip-10-101-0-15 ~]$
[ec2-user@ip-10-101-0-15 ~]$ ssh -i rubin-stg-cn-master.pem ec2-user@10.101.3.236
The authenticity of host '10.101.3.236 (10.101.3.236)' can't be established.
ECDSA key fingerprint is SHA256:0hRgvNmSwRB8j0GFN9c5So1cs6btMn8Wgqb9w4tQgWU.
ECDSA key fingerprint is MD5:ba:75:a7:ba:35:22:af:8f:49:a9:cc:e7:d4:b4:3b:40.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.101.3.236' (ECDSA) to the list of known hosts.
__| __|_ )
_| ( / Amazon Linux 2 AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-2/
6 package(s) needed for security, out of 337 available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-10-101-3-236 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
link/ether 02:58:2c:5d:73:94 brd ff:ff:ff:ff:ff:ff
inet 10.101.3.236/24 brd 10.101.3.255 scope global dynamic eth0
valid_lft 3014sec preferred_lft 3014sec
inet6 fe80::58:2cff:fe5d:7394/64 scope link
valid_lft forever preferred_lft forever
[ec2-user@ip-10-101-3-236 ~]$ curl ifconfig.me #查看出口IP是创建的NAT Gateway的IP
54.222.176.3
[ec2-user@ip-10-101-3-236 ~]$ curl -I www.baidu.com
HTTP/1.1 200 OK
Accept-Ranges: bytes
Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
Connection: Keep-Alive
Content-Length: 277
Content-Type: text/html
Date: Mon, 22 Oct 2018 13:57:37 GMT
Etag: "575e1f60-115"
Last-Modified: Mon, 13 Jun 2016 02:50:08 GMT
Pragma: no-cache
Server: bfe/1.0.8.18
[ec2-user@ip-10-101-3-236 ~]$
[ec2-user@ip-10-101-3-236 ~]$ nslookup www.baidu.com
Server: 10.101.0.2
Address: 10.101.0.2#53
Non-authoritative answer:
www.baidu.com canonical name = www.a.shifen.com.
Name: www.a.shifen.com
Address: 220.181.112.244
Name: www.a.shifen.com
Address: 220.181.111.188
