开源IaC工具 - Terraform

目录

一.Terraform简介

二、Terraform核心概念

Provider:与外部系统交互的插件

Resource:最小管理单元

State:Terraform的"单一事实源"

Module:实现代码复用和架构分层的核心机制

三、Terraform标准目录结构

四、Terraform内部执行流程

[1.加载Root Module](#1.加载Root Module)

2.Backend与State初始化

3.Provider初始化与校验

4.构建资源依赖图(DAG)

5.Plan(变更计算)

6.Apply(真正的API调用)

7.State回写与锁释放

五、Terraform多云环境架构示例

场景描述

配置示例

[1.配置Terraform Cloud Workspaces](#1.配置Terraform Cloud Workspaces)

[2.配置aws client和aliyun client](#2.配置aws client和aliyun client)

3.配置terraform项目

4.创建测试项目

5.创建资源

6.查看资源


一.Terraform简介

IaC(基础设施即代码)是一种通过代码来定义、部署和管理基础设施的方式,可以代替手工在控制台的各种点击操作。在IaC模式下,以下资源都可以被"写成代码"并版本化管理:

  1. 云资源:VPC、子网、ECS/EC2、RDS、负载均衡、对象存储等

  2. 平台资源:Kubernetes、Node、Namespace、Ingress等

IaC主要为解决以下问题:

  1. 环境不容易复现

  2. 运维变更无记录、难回滚

  3. 各环境(dev/test/prod)配置不一致

  4. 大规模资源维护成本极高

Terraform是HashiCorp推出的、当前最主流的IaC工具之一。它通过声明式配置语言(HCL) 来描述"期望的基础设施状态",Terraform会自动计算当前状态与期望状态的差异,并执行变更。

需要特别强调的是:

1.Terraform不是配置管理工具(不负责装软件)

2.Terraform不是云平台的抽象替代

3.Terraform是基础设施生命周期管理工具

二、Terraform核心概念

Provider:与外部系统交互的插件

Provider的本质是将Terraform的资源模型,翻译为具体云厂商API调用。Terraform所有 CRUD行为都由Provider完成;Provider决定了apply阶段的真实行为。

Resource:最小管理单元

Resource的重要特性:只关心资源ID;不存在ID就是不存在资源;ID的映射关系保存在state 中。

State:Terraform的"单一事实源"

这是Terraform最重要、也最容易被低估的部分。Terraform不会每次都去云平台全量扫描资源,而是以state文件作为基础设施的"快照"。State中保存了:资源与真实云ID的映射;上一次 apply的属性值;Provider元信息。

尽量不使用local state,而是使用集中存储(如S3、Terraform Cloud等)。

Module:实现代码复用和架构分层的核心机制

Module的作用不是"复用几行代码",而是:把基础设施能力封装成稳定接口。正确高效的Module设计原则:只描述"能力",不关心环境;不写 backend;不区分dev/prod等。

三、Terraform标准目录结构

Terraform目录结构核心原则:

1.一个环境使用一个state

2.只在环境目录执行命令

  1. Module只负责描述基础设施能力,不参与环境实例化与状态管理。

以下是我们今天的示例:

+---envs

| +---dev

| | | main.tf

| | | outputs.tf

| | | provider.tf

| | | terraform.tf

| | | terraform.tfvars

| | | variables.tf

| +---test

| | main.tf

| | outputs.tf

| | provider.tf

| | terraform.tf

| | terraform.tfvars

| | variables.tf

+---modules

+---aliyun-oss

| main.tf

| outputs.tf

| terraform.tf

| variables.tf

+---aws-s3

main.tf

outputs.tf

variables.tf

四、Terraform内部执行流程

下面按真实执行顺序拆解terraform apply:

1.加载Root Module

读取当前目录所有 *.tf

合并为一个逻辑模块

文件名不影响执行顺序

2.Backend与State初始化

连接远端backend(OSS/S3/Terraform Cloud)

下载state到内存

加锁,防止并发apply

3.Provider初始化与校验

校验provider版本

校验认证信息

4.构建资源依赖图(DAG)

Terraform会根据属性引用、depends_on来构建有向无环图,决定:

创建/更新/删除顺序

哪些资源可以并发

5.Plan(变更计算)

Terraform对比当前配置和当前state,输出三种行为:

  • Create

~ Update

  • Destroy

6.Apply(真正的API调用)

这是唯一"危险阶段"。可能出现的情况:正常创建、API超时、配额不足、网络中断等。

7.State回写与锁释放

成功:state更新并写回

失败:可能产生漂移

五、Terraform多云环境架构示例

场景描述

  • AWS:海外业务

  • 阿里云:国内业务

  • Dev环境:创建AWS S3资源、阿里云OSS资源

  • Test环境:只创建AWS S3资源

配置示例

1.配置Terraform Cloud Workspaces

示例中我们使用Terraform Cloud作为远端State。登录https://app.terraform.io/

创建Organization

创建Projects(或者使用默认),并做设置

创建Workspaces:dev-workspace和test-workspace

点击右上角Account settings,创建登录Tokens

2.配置aws client和aliyun client

登录aws ,新建IAM用户,授予S3管理权限,并创建Access key

登录aliyun,新建RAM用户,授予OSS管理权限,并创建Access key。设置网络访问限制策略为"允许所有访问"。

在自己电脑下载安装client,并设置环境变量

https://awscli.amazonaws.com/AWSCLIV2.msi
https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip

分别执行aws configure和aliyun configure,并输入各自的Access key信息;然后检查状态:

3.配置terraform项目

在自己电脑下载安装terraform客户端

https://releases.hashicorp.com/terraform/1.14.2/terraform_1.14.2_windows_amd64.zip

IDEA中设置terraform

4.创建测试项目

如标准目录结构所示,我们创建envs/dev和envs/test来分别管理Dev环境和Test环境;创建modules/aliyun-oss和modules/aws-s3分别描述阿里云资源和AWS资源。

modules相关文件内容如下:

复制代码
#modules/aliyun-oss/main.tf
resource "alicloud_oss_bucket" "this" {
  bucket = var.bucket_name
  tags   = var.tags
}

#modules/aliyun-oss/outputs.tf
output "bucket_name" {
  value = alicloud_oss_bucket.this.bucket
}
output "bucket_id" {
  value = alicloud_oss_bucket.this.id
}

#modules/aliyun-oss/terraform.tf
#此文件为解决aliyun插件版本遗留问题
terraform {
  required_providers {
    alicloud = {
      source  = "aliyun/alicloud"
      version = "~> 1.266.0"
    }
  }
}

#modules/aliyun-oss/variables.tf
variable "bucket_name" {
  type        = string
  description = "OSS bucket name"
}
variable "acl" {
  type        = string
  default     = "private"
}
variable "tags" {
  type        = map(string)
  default     = {}
}

#modules/aws-s3/main.tf
resource "aws_s3_bucket" "this" {
  bucket = var.bucket_name
  tags   = var.tags
}

#modules/aws-s3/outputs.tf
output "bucket_name" {
  value = aws_s3_bucket.this.bucket
}

#modules/aws-s3/variables.tf
variable "bucket_name" {
  description = "S3 bucket name"
  type        = string
}
variable "tags" {
  description = "Common tags"
  type        = map(string)
  default     = {}
}

envs/dev相关文件内容如下:

复制代码
#envs/dev/main.tf
module "aws_s3" {
  source = "../../modules/aws-s3"

  for_each = toset(var.s3_buckets)
  bucket_name = each.value
  tags        = var.common_tags
}
module "aliyun_oss" {
  source = "../../modules/aliyun-oss"
  for_each    = toset(var.aliyun_oss_buckets)
  bucket_name = each.value
  tags        = var.common_tags
}

#envs/dev/outputs.tf
output "aws_s3_buckets" {
  description = "All S3 bucket names created in this environment"
  value = {
    for k, m in module.aws_s3 :
    k => m.bucket_name
  }
}
output "aliyun_oss_buckets" {
  value = {
    for k, m in module.aliyun_oss :
    k => m.bucket_name
  }
}

#envs/dev/provider.tf
provider "aws" {
  region = "ap-southeast-1"
}
provider "alicloud" {
  region = "cn-hangzhou"
  profile = "default"
}

#envs/dev/terraform.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.27.0"
    }
    alicloud = {
      source  = "aliyun/alicloud"
      version = "~> 1.266.0"
    }
  }
  cloud {
    organization = "my-test-companys"

    workspaces {
      name = "dev-workspace"
    }
  }
}

#envs/dev/terraform.tfvars
s3_buckets = [
  "dev-devops-s3-demo",
  "dev-devops-s3-demo2"
]
aliyun_oss_buckets = [
  "dev-devops-aliyun-demo",
  "dev-devops-aliyun-demo2"
]
common_tags = {
  Environment = "dev"
  Owner       = "platform-team"
  ManagedBy   = "terraform"
}

#envs/dev/variables.tf
variable "s3_buckets" {
  description = "List of S3 buckets to create"
  type        = list(string)
}
variable "common_tags" {
  description = "Common tags for all resources"
  type        = map(string)
}
variable "aliyun_oss_buckets" {
  type = list(string)
}

envs/test相关文件内容如下:

复制代码
#envs/test/main.tf
module "s3" {
  source = "../../modules/aws-s3"

  for_each = toset(var.s3_buckets)

  bucket_name = each.value
  tags        = var.common_tags
}


#envs/test/outputs.tf
output "aws_s3_buckets" {
  description = "All S3 bucket names created in this environment"
  value = {
    for k, m in module.s3 :
    k => m.bucket_name
  }
}


#envs/test/provider.tf
provider "aws" {
  region = "ap-southeast-1"
}

#envs/test/terraform.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.27.0"
    }
  }

  cloud {
    organization = "my-test-companys"

    workspaces {
      name = "test-workspace"
    }
  }
}


#envs/test/terraform.tfvars
s3_buckets = [
  "test-devops-s3-demo",
  "test-devops-s3-demo2"
]

common_tags = {
  Environment = "test"
  Owner       = "platform-team"
  ManagedBy   = "terraform"
}


#envs/test/variables.tf
variable "s3_buckets" {
  description = "List of S3 buckets to create"
  type        = list(string)
}

variable "common_tags" {
  description = "Common tags for all resources"
  type        = map(string)
}

5.创建资源

分别到envs/dev目录和envs/test目录下执行命令

PS D:\个人\HCL> cd .\envs\dev\

PS D:\个人\HCL\envs\dev> terraform init

Initializing HCP Terraform...

Initializing modules...

Initializing provider plugins...

  • Reusing previous version of aliyun/alicloud from the dependency lock file

  • Reusing previous version of hashicorp/aws from the dependency lock file

  • Using previously-installed aliyun/alicloud v1.266.0

  • Using previously-installed hashicorp/aws v6.27.0

HCP Terraform has been successfully initialized!

You may now begin working with HCP Terraform. Try running "terraform plan" to

see any changes that are required for your infrastructure.

If you ever set or change modules or Terraform Settings, run "terraform init"

again to reinitialize your working directory.

PS D:\个人\HCL\envs\dev> terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

  • create

Terraform will perform the following actions:

module.aliyun_oss["dev-devops-aliyun-demo"].alicloud_oss_bucket.this will be created

  • resource "alicloud_oss_bucket" "this" {

  • acl = (known after apply)

  • bucket = "dev-devops-aliyun-demo"

  • creation_date = (known after apply)

  • extranet_endpoint = (known after apply)

  • force_destroy = false

  • id = (known after apply)

  • intranet_endpoint = (known after apply)

  • lifecycle_rule_allow_same_action_overlap = false

  • location = (known after apply)

  • owner = (known after apply)

  • redundancy_type = "LRS"

  • resource_group_id = (known after apply)

  • storage_class = "Standard"

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • access_monitor (known after apply)

}

module.aliyun_oss["dev-devops-aliyun-demo2"].alicloud_oss_bucket.this will be created

  • resource "alicloud_oss_bucket" "this" {

  • acl = (known after apply)

  • bucket = "dev-devops-aliyun-demo2"

  • creation_date = (known after apply)

  • extranet_endpoint = (known after apply)

  • force_destroy = false

  • id = (known after apply)

  • intranet_endpoint = (known after apply)

  • lifecycle_rule_allow_same_action_overlap = false

  • location = (known after apply)

  • owner = (known after apply)

  • redundancy_type = "LRS"

  • resource_group_id = (known after apply)

  • storage_class = "Standard"

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • access_monitor (known after apply)

}

module.aws_s3["dev-devops-s3-demo"].aws_s3_bucket.this will be created

  • resource "aws_s3_bucket" "this" {

  • acceleration_status = (known after apply)

  • acl = (known after apply)

  • arn = (known after apply)

  • bucket = "dev-devops-s3-demo"

  • bucket_domain_name = (known after apply)

  • bucket_prefix = (known after apply)

  • bucket_region = (known after apply)

  • bucket_regional_domain_name = (known after apply)

  • force_destroy = false

  • hosted_zone_id = (known after apply)

  • id = (known after apply)

  • object_lock_enabled = (known after apply)

  • policy = (known after apply)

  • region = "ap-southeast-1"

  • request_payer = (known after apply)

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • tags_all = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • website_domain = (known after apply)

  • website_endpoint = (known after apply)

  • cors_rule (known after apply)

  • grant (known after apply)

  • lifecycle_rule (known after apply)

  • logging (known after apply)

  • object_lock_configuration (known after apply)

  • replication_configuration (known after apply)

  • server_side_encryption_configuration (known after apply)

  • versioning (known after apply)

  • website (known after apply)

}

module.aws_s3["dev-devops-s3-demo2"].aws_s3_bucket.this will be created

  • resource "aws_s3_bucket" "this" {

  • acceleration_status = (known after apply)

  • acl = (known after apply)

  • arn = (known after apply)

  • bucket = "dev-devops-s3-demo2"

  • bucket_domain_name = (known after apply)

  • bucket_prefix = (known after apply)

  • bucket_region = (known after apply)

  • bucket_regional_domain_name = (known after apply)

  • force_destroy = false

  • hosted_zone_id = (known after apply)

  • id = (known after apply)

  • object_lock_enabled = (known after apply)

  • policy = (known after apply)

  • region = "ap-southeast-1"

  • request_payer = (known after apply)

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • tags_all = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • website_domain = (known after apply)

  • website_endpoint = (known after apply)

  • cors_rule (known after apply)

  • grant (known after apply)

  • lifecycle_rule (known after apply)

  • logging (known after apply)

  • object_lock_configuration (known after apply)

  • replication_configuration (known after apply)

  • server_side_encryption_configuration (known after apply)

  • versioning (known after apply)

  • website (known after apply)

}

Plan: 4 to add, 0 to change, 0 to destroy.

Changes to Outputs:

~ aliyun_oss_buckets = {

  • dev-devops-aliyun-demo = "dev-devops-aliyun-demo"

  • dev-devops-aliyun-demo2 = "dev-devops-aliyun-demo2"

}

~ aws_s3_buckets = {

  • dev-devops-s3-demo = "dev-devops-s3-demo"

  • dev-devops-s3-demo2 = "dev-devops-s3-demo2"

}

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

Releasing state lock. This may take a few moments...

PS D:\个人\HCL\envs\dev> terraform apply -auto-approve

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

  • create

Terraform will perform the following actions:

module.aliyun_oss["dev-devops-aliyun-demo"].alicloud_oss_bucket.this will be created

  • resource "alicloud_oss_bucket" "this" {

  • acl = (known after apply)

  • bucket = "dev-devops-aliyun-demo"

  • creation_date = (known after apply)

  • extranet_endpoint = (known after apply)

  • force_destroy = false

  • id = (known after apply)

  • intranet_endpoint = (known after apply)

  • lifecycle_rule_allow_same_action_overlap = false

  • location = (known after apply)

  • owner = (known after apply)

  • redundancy_type = "LRS"

  • resource_group_id = (known after apply)

  • storage_class = "Standard"

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • access_monitor (known after apply)

}

module.aliyun_oss["dev-devops-aliyun-demo2"].alicloud_oss_bucket.this will be created

  • resource "alicloud_oss_bucket" "this" {

  • acl = (known after apply)

  • bucket = "dev-devops-aliyun-demo2"

  • creation_date = (known after apply)

  • extranet_endpoint = (known after apply)

  • force_destroy = false

  • id = (known after apply)

  • intranet_endpoint = (known after apply)

  • lifecycle_rule_allow_same_action_overlap = false

  • location = (known after apply)

  • owner = (known after apply)

  • redundancy_type = "LRS"

  • resource_group_id = (known after apply)

  • storage_class = "Standard"

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • access_monitor (known after apply)

}

module.aws_s3["dev-devops-s3-demo"].aws_s3_bucket.this will be created

  • resource "aws_s3_bucket" "this" {

  • acceleration_status = (known after apply)

  • acl = (known after apply)

  • arn = (known after apply)

  • bucket = "dev-devops-s3-demo"

  • bucket_domain_name = (known after apply)

  • bucket_prefix = (known after apply)

  • bucket_region = (known after apply)

  • bucket_regional_domain_name = (known after apply)

  • force_destroy = false

  • hosted_zone_id = (known after apply)

  • id = (known after apply)

  • object_lock_enabled = (known after apply)

  • policy = (known after apply)

  • region = "ap-southeast-1"

  • request_payer = (known after apply)

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • tags_all = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • website_domain = (known after apply)

  • website_endpoint = (known after apply)

  • cors_rule (known after apply)

  • grant (known after apply)

  • lifecycle_rule (known after apply)

  • logging (known after apply)

  • object_lock_configuration (known after apply)

  • replication_configuration (known after apply)

  • server_side_encryption_configuration (known after apply)

  • versioning (known after apply)

  • website (known after apply)

}

module.aws_s3["dev-devops-s3-demo2"].aws_s3_bucket.this will be created

  • resource "aws_s3_bucket" "this" {

  • acceleration_status = (known after apply)

  • acl = (known after apply)

  • arn = (known after apply)

  • bucket = "dev-devops-s3-demo2"

  • bucket_domain_name = (known after apply)

  • bucket_prefix = (known after apply)

  • bucket_region = (known after apply)

  • bucket_regional_domain_name = (known after apply)

  • force_destroy = false

  • hosted_zone_id = (known after apply)

  • id = (known after apply)

  • object_lock_enabled = (known after apply)

  • policy = (known after apply)

  • region = "ap-southeast-1"

  • request_payer = (known after apply)

  • tags = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • tags_all = {

  • "Environment" = "dev"

  • "ManagedBy" = "terraform"

  • "Owner" = "platform-team"

}

  • website_domain = (known after apply)

  • website_endpoint = (known after apply)

  • cors_rule (known after apply)

  • grant (known after apply)

  • lifecycle_rule (known after apply)

  • logging (known after apply)

  • object_lock_configuration (known after apply)

  • replication_configuration (known after apply)

  • server_side_encryption_configuration (known after apply)

  • versioning (known after apply)

  • website (known after apply)

}

Plan: 4 to add, 0 to change, 0 to destroy.

Changes to Outputs:

~ aliyun_oss_buckets = {

  • dev-devops-aliyun-demo = "dev-devops-aliyun-demo"

  • dev-devops-aliyun-demo2 = "dev-devops-aliyun-demo2"

}

~ aws_s3_buckets = {

  • dev-devops-s3-demo = "dev-devops-s3-demo"

  • dev-devops-s3-demo2 = "dev-devops-s3-demo2"

}

module.aliyun_oss["dev-devops-aliyun-demo"].alicloud_oss_bucket.this: Creating...

module.aliyun_oss["dev-devops-aliyun-demo2"].alicloud_oss_bucket.this: Creating...

module.aws_s3["dev-devops-s3-demo"].aws_s3_bucket.this: Creating...

module.aws_s3["dev-devops-s3-demo2"].aws_s3_bucket.this: Creating...

module.aws_s3["dev-devops-s3-demo"].aws_s3_bucket.this: Creation complete after 3s [id=dev-devops-s3-demo]

module.aws_s3["dev-devops-s3-demo2"].aws_s3_bucket.this: Creation complete after 3s [id=dev-devops-s3-demo2]

module.aliyun_oss["dev-devops-aliyun-demo"].alicloud_oss_bucket.this: Creation complete after 4s [id=dev-devops-aliyun-demo]

module.aliyun_oss["dev-devops-aliyun-demo2"].alicloud_oss_bucket.this: Creation complete after 4s [id=dev-devops-aliyun-demo2]

Releasing state lock. This may take a few moments...

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

aliyun_oss_buckets = {

"dev-devops-aliyun-demo" = "dev-devops-aliyun-demo"

"dev-devops-aliyun-demo2" = "dev-devops-aliyun-demo2"

}

aws_s3_buckets = {

"dev-devops-s3-demo" = "dev-devops-s3-demo"

"dev-devops-s3-demo2" = "dev-devops-s3-demo2"

}

PS D:\个人\HCL\envs\dev> cd ..\test\
PS D:\个人\HCL\envs\test>【测试环境执行命令同上】

6.查看资源

登录AWS和Aliyun账号,看到资源已经创建

查看Terraform Cloud Workspaces中的记录

相关推荐
梦帮科技2 小时前
第二十三篇:自然语言工作流生成:GPT-4集成实战
人工智能·python·机器学习·开源·gpt-3·极限编程
IvorySQL2 小时前
Postgres 18 默认开启数据校验及升级应对方案
数据库·人工智能·postgresql·开源
qq_463408422 小时前
React Native跨平台技术在开源鸿蒙中使用内置的`fetch` API或者第三方库如`axHarmony`来处理网络通信HTTP请求
javascript·算法·react native·react.js·http·开源·harmonyos
青年夏日科技工作者2 小时前
docker运行debian / Ubuntu桌面容器 xrdp(带声音) x11 vnc novnc
云原生·eureka
想学后端的前端工程师2 小时前
【分布式系统架构设计实战:从单体到微服务】
微服务·云原生·架构
忍冬行者2 小时前
kubernetes安装traefik ingress,替换原来的nginx-ingress
云原生·容器·kubernetes
Lynnxiaowen2 小时前
今天我们继续DevOps内容Jenkins自动化部署PHP项目
linux·自动化·jenkins·php·devops
篙芷2 小时前
k8s Service 暴露方式详解:ClusterIP、NodePort、LoadBalancer 与 Headless Service
云原生·容器·kubernetes
aashuii2 小时前
k8s POD上RDMA网卡VF不生效问题
云原生·容器·kubernetes