Advanced Module Patterns for Enterprise Scale

The day our monolithic Terraform repo took 45 minutes to plan taught me that module architecture matters

Introduction: The Monolith Problem

It was a Thursday afternoon. Our lead developer ran terraform plan on our production infrastructure repository.

We waited.

And waited.

45 minutes later, the plan finally completed.

The problem? Everything was in one massive Terraform configuration:

200+ resources
No module boundaries
Tight coupling
Circular dependencies
Impossible to test
Dangerous to change

Every small change required planning the entire infrastructure. Every deployment was risky. The team was afraid to make changes.

We spent the next two months refactoring into a modular architecture:

Before: 45-minute plans, 1-hour deployments
After: 2-minute plans, 5-minute deployments

I learned:

Module boundaries prevent tight coupling
Composition enables complexity management
Versioning allows independent evolution
Repository structure impacts team productivity
Good architecture scales with your organization

This article is everything I learned about building complex, maintainable Terraform architectures.

Why Module Architecture Matters

The Cost of Poor Architecture

Symptoms:

⏱️ Slow terraform plan
😰 Fear of making changes
🐛 Frequent breakages
🔄 Circular dependencies
📉 Low team productivity
🚫 Difficult to test

Impact:

Deployment delays
Increased risk
Reduced innovation
Team frustration

Benefits of Good Architecture

Well-designed modules enable:

✅ Fast feedback loops
✅ Independent deployment
✅ Easy testing
✅ Clear ownership
✅ Reusability
✅ Scalability

Architecture Evolution

Module Design Principles

1. Single Responsibility

Each module should do one thing well.

❌ Bad:

module "everything" {
  # Database
  # Application
  # Monitoring
  # Networking
  # ...
}

✅ Good:

module "database" {
  # Just database
}

module "application" {
  # Just application
}

module "monitoring" {
  # Just monitoring
}

2. Clear Interface

Inputs and outputs should be well-defined and documented.

❌ Bad:

variable "config" {
  type = any  # Unclear what's expected
}

✅ Good:

variable "database_config" {
  type = object({
    engine         = string
    version        = string
    instance_class = string
    storage_gb     = number
  })
  description = "Database configuration"
  
  validation {
    condition     = contains(["postgres", "mysql"], var.database_config.engine)
    error_message = "Engine must be postgres or mysql."
  }
}

3. Loose Coupling

Modules should minimize dependencies.

❌ Tight Coupling:

# Application module directly accesses database module internals
resource "local_file" "app_config" {
  content = module.database.internal_connection_string
}

✅ Loose Coupling:

# Application module uses well-defined outputs
resource "local_file" "app_config" {
  content = module.database.connection_endpoint
}

4. High Cohesion

Related functionality should be grouped together.

✅ Good cohesion:

database-module/
├── main.tf           # Database resources
├── backup.tf         # Database backups
├── monitoring.tf     # Database monitoring
├── variables.tf
└── outputs.tf

5. Composition Over Inheritance

Build complex modules from simple ones.

# Simple modules
module "database" { }
module "cache" { }
module "storage" { }

# Composed module
module "application_stack" {
  database = module.database
  cache    = module.cache
  storage  = module.storage
}

Complex Module Patterns

Pattern 1: Layered Architecture

Separate infrastructure into logical layers.

infrastructure/
├── foundation/          # Layer 1: Networking, DNS
│   ├── network/
│   └── dns/
├── data/               # Layer 2: Databases, Caches
│   ├── database/
│   └── cache/
├── application/        # Layer 3: Services
│   ├── web/
│   ├── api/
│   └── worker/
└── monitoring/         # Layer 4: Observability
    ├── logs/
    └── metrics/

Example:

foundation/network/main.tf:

resource "local_file" "network_config" {
  filename = "network.yaml"
  content  = yamlencode({
    vpc_cidr    = var.vpc_cidr
    subnets     = var.subnets
    enable_nat  = var.enable_nat
  })
}

output "network_config" {
  value = local_file.network_config.filename
}

data/database/main.tf:

variable "network_config" {
  type        = string
  description = "Path to network configuration"
}

data "local_file" "network" {
  filename = var.network_config
}

locals {
  network = yamldecode(data.local_file.network.content)
}

resource "local_file" "database_config" {
  filename = "database.yaml"
  content  = yamlencode({
    network_info = local.network
    database = {
      engine  = var.engine
      version = var.version
    }
  })
}

Root composition:

module "network" {
  source   = "./foundation/network"
  vpc_cidr = "10.0.0.0/16"
}

module "database" {
  source         = "./data/database"
  network_config = module.network.network_config
  engine         = "postgres"
  version        = "14"
}

Pattern 2: Feature Modules

Group related features together.

modules/
├── blog-platform/
│   ├── database.tf      # Blog database
│   ├── storage.tf       # Media storage
│   ├── cdn.tf          # CDN for assets
│   └── search.tf       # Search index
├── auth-service/
│   ├── database.tf
│   ├── cache.tf
│   └── api.tf
└── analytics/
    ├── collector.tf
    ├── processor.tf
    └── dashboard.tf

Pattern 3: Environment Wrapper

Wrap modules with environment-specific configuration.

modules/
└── service/             # Generic service module
    ├── main.tf
    ├── variables.tf
    └── outputs.tf

environments/
├── dev/
│   └── service/
│       └── main.tf      # Dev-specific config
├── staging/
│   └── service/
│       └── main.tf      # Staging-specific config
└── production/
    └── service/
        └── main.tf      # Production-specific config

environments/production/service/main.tf:

module "service" {
  source = "../../../modules/service"
  
  environment     = "production"
  instance_count  = 5
  instance_size   = "large"
  backup_enabled  = true
  monitoring      = "enhanced"
}

Pattern 4: Factory Pattern

Generate multiple similar resources.

variable "services" {
  type = map(object({
    port        = number
    replicas    = number
    environment = map(string)
  }))
}

module "service" {
  for_each = var.services
  source   = "./modules/service"
  
  name        = each.key
  port        = each.value.port
  replicas    = each.value.replicas
  environment = each.value.environment
}

Usage:

services = {
  web = {
    port     = 8080
    replicas = 3
    environment = {
      LOG_LEVEL = "info"
    }
  }
  api = {
    port     = 8081
    replicas = 5
    environment = {
      LOG_LEVEL = "debug"
    }
  }
}

Pattern 5: Builder Pattern

Progressively build complex configurations.

module "base_service" {
  source = "./modules/base-service"
  name   = "my-service"
}

module "with_database" {
  source = "./modules/add-database"
  
  service_config = module.base_service.config
  database_type  = "postgres"
}

module "with_cache" {
  source = "./modules/add-cache"
  
  service_config = module.with_database.config
  cache_type     = "redis"
}

module "with_monitoring" {
  source = "./modules/add-monitoring"
  
  service_config = module.with_cache.config
  alert_email    = "[email protected]"
}

Module Composition Strategies

Strategy 1: Nested Composition

Modules call other modules.

# Parent module
module "database_cluster" {
  source = "./database-cluster"
}

# database-cluster module calls child modules
module "primary" {
  source = "../database-instance"
  role   = "primary"
}

module "replica" {
  source = "../database-instance"
  count  = var.replica_count
  role   = "replica"
}

Strategy 2: Sibling Composition

Modules interact through outputs.

module "database" {
  source = "./modules/database"
}

module "application" {
  source = "./modules/application"
  
  database_host = module.database.host
  database_port = module.database.port
}

module "monitoring" {
  source = "./modules/monitoring"
  
  database_endpoint    = module.database.endpoint
  application_endpoint = module.application.endpoint
}

Strategy 3: Registry Composition

Use external modules from registries.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.0.0"
  
  name = "production-vpc"
  cidr = "10.0.0.0/16"
}

module "database" {
  source = "./modules/custom-database"
  
  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.database_subnets
}

Strategy 4: Conditional Composition

Include modules conditionally.

variable "enable_monitoring" {
  type    = bool
  default = false
}

module "monitoring" {
  count  = var.enable_monitoring ? 1 : 0
  source = "./modules/monitoring"
  
  service_endpoint = module.application.endpoint
}

Strategy 5: Data-Driven Composition

Generate modules from data.

locals {
  services = yamldecode(file("services.yaml"))
}

module "service" {
  for_each = local.services
  source   = "./modules/service"
  
  name   = each.key
  config = each.value
}

services.yaml:

web:
  port: 8080
  replicas: 3
api:
  port: 8081
  replicas: 5
worker:
  port: 8082
  replicas: 2

Mono-repo vs Multi-repo

Mono-repo Architecture

All infrastructure in one repository.

terraform-monorepo/
├── modules/
│   ├── networking/
│   ├── database/
│   ├── application/
│   └── monitoring/
├── environments/
│   ├── dev/
│   ├── staging/
│   └── production/
├── policies/
└── docs/

Advantages: ✅ Single source of truth ✅ Easy cross-module refactoring ✅ Simplified versioning ✅ Easier to enforce standards ✅ Atomic changes across modules

Disadvantages: ❌ Large repository size ❌ Slower CI/CD ❌ Harder access control ❌ Potential merge conflicts ❌ Scales poorly with team size

Multi-repo Architecture

Modules in separate repositories.

terraform-networking/       (repo 1)
terraform-database/         (repo 2)
terraform-application/      (repo 3)
terraform-monitoring/       (repo 4)
terraform-environments/     (repo 5)

Advantages: ✅ Clear ownership boundaries ✅ Independent versioning ✅ Faster CI/CD per repo ✅ Better access control ✅ Scales with teams

Disadvantages: ❌ Coordination overhead ❌ Version management complexity ❌ Harder to refactor across modules ❌ Discovery challenges ❌ Dependency tracking

Hybrid Approach

Best of both worlds:

terraform-core/             (mono-repo)
├── foundation-modules/
└── shared-modules/

terraform-team-a/           (team repo)
└── team-a-services/

terraform-team-b/           (team repo)
└── team-b-services/

Use mono-repo for:

Foundation modules
Shared libraries
Company standards

Use multi-repo for:

Team-specific modules
Application infrastructure
Experimental work

Decision Matrix

Factor

Mono-repo

Multi-repo

Team Size

Small (<10)

Large (>10)

Change Frequency

High coupling

Low coupling

Access Control

Same for all

Different per module

Deployment

Together

Independent

Discoverability

Easy

Harder

Module Versioning Strategies

Semantic Versioning

Format: MAJOR.MINOR.PATCH

MAJOR: Breaking changes
MINOR: New features (backward compatible)
PATCH: Bug fixes

Example:

module "database" {
  source  = "git::https://github.com/org/terraform-database.git?ref=v2.3.1"
  # v2 = Major version
  # 3 = Minor version
  # 1 = Patch version
}

Git Tags

# Create version tag
git tag -a v1.0.0 -m "Release v1.0.0"
git push origin v1.0.0

# Use in Terraform
module "database" {
  source = "git::https://github.com/org/terraform-database.git?ref=v1.0.0"
}

Version Constraints

module "database" {
  source  = "terraform-aws-modules/rds/aws"
  version = "~> 5.0"  # 5.0 <= version < 6.0
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = ">= 4.0, < 5.0"  # Range
}

module "custom" {
  source  = "git::https://github.com/org/module.git?ref=v1.2.3"
  # Exact version
}

Version Pinning

❌ Risky:

module "database" {
  source = "git::https://github.com/org/module.git?ref=main"
  # Uses latest, may break
}

✅ Safe:

module "database" {
  source = "git::https://github.com/org/module.git?ref=v1.2.3"
  # Pinned to specific version
}

Terraform Registry Versions

Private registry (terraform.tfvars):

module "internal_service" {
  source  = "app.terraform.io/my-org/service/aws"
  version = "2.1.0"
}

Version Update Strategy

1. Test in dev:

# environments/dev/main.tf
module "database" {
  source  = "git::https://github.com/org/module.git?ref=v2.0.0"
  # Test new version
}

2. Promote to staging:

# environments/staging/main.tf
module "database" {
  source  = "git::https://github.com/org/module.git?ref=v2.0.0"
  # Same version as dev
}

3. Deploy to production:

# environments/production/main.tf
module "database" {
  source  = "git::https://github.com/org/module.git?ref=v2.0.0"
  # Promoted version
}

Module Registry Best Practices

Publishing Modules

Directory structure:

terraform-aws-service/
├── main.tf
├── variables.tf
├── outputs.tf
├── README.md
├── LICENSE
├── examples/
│   ├── basic/
│   └── complete/
└── tests/
    └── service_test.go

README.md template:

# AWS Service Module

## Description
Brief description of what this module does.

## Usage
```hcl
module "service" {
  source = "github.com/org/terraform-aws-service"
  
  name        = "my-service"
  environment = "production"
}

Requirements

Terraform >= 1.6.0
AWS Provider >= 5.0

Inputs

Name

Description

Type

Default

Required

name

Service name

string

n/a

yes

Outputs

Name

Description

endpoint

Service endpoint

Examples

See examples/ directory.

License

MIT


### Terraform Registry Requirements

1. **GitHub repository** named `terraform-<PROVIDER>-<NAME>`
2. **Description** in GitHub repo
3. **Git tags** for versions (v1.0.0)
4. **Standard structure** (main.tf, variables.tf, outputs.tf)
5. **README.md** with usage examples
6. **LICENSE** file

**Example repository:**

Repository: terraform-aws-database Description: AWS database module Tags: v1.0.0, v1.1.0, v2.0.0


### Private Registry

**Terraform Cloud:**

1. **Create module:**
   - Organization settings → Registry → Publish → Module
   - Connect GitHub repository
   - Select repository
   - Terraform Cloud watches for new tags

2. **Use module:**
```hcl
module "database" {
  source  = "app.terraform.io/my-org/database/aws"
  version = "1.0.0"
}

Module Documentation

Auto-generated docs:

# Install terraform-docs
brew install terraform-docs

# Generate README
terraform-docs markdown table . > README.md

Example output:

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_name"></a> [name](#input\_name) | Service name | `string` | n/a | yes |
| <a name="input_port"></a> [port](#input\_port) | Service port | `number` | `8080` | no |

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_endpoint"></a> [endpoint](#output\_endpoint) | Service endpoint |

Testing Complex Modules

Unit Testing Modules

modules/service/test/service_test.go:

package test

import (
	"testing"
	"github.com/gruntwork-io/terratest/modules/terraform"
	"github.com/stretchr/testify/assert"
)

func TestServiceModule(t *testing.T) {
	t.Parallel()

	terraformOptions := &terraform.Options{
		TerraformDir: "../",
		Vars: map[string]interface{}{
			"service_name": "test-service",
			"port":         8080,
			"environment":  "test",
		},
	}

	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)

	// Test outputs
	endpoint := terraform.Output(t, terraformOptions, "endpoint")
	assert.Contains(t, endpoint, "test-service")
	assert.Contains(t, endpoint, "8080")
}

Integration Testing

Test module composition:

func TestServiceWithDatabase(t *testing.T) {
	t.Parallel()

	terraformOptions := &terraform.Options{
		TerraformDir: "../fixtures/complete",
		Vars: map[string]interface{}{
			"environment": "test",
		},
	}

	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)

	// Verify database module
	dbEndpoint := terraform.Output(t, terraformOptions, "database_endpoint")
	assert.NotEmpty(t, dbEndpoint)

	// Verify application module
	appEndpoint := terraform.Output(t, terraformOptions, "application_endpoint")
	assert.NotEmpty(t, appEndpoint)

	// Verify integration
	config := terraform.Output(t, terraformOptions, "application_config")
	assert.Contains(t, config, dbEndpoint)
}

Contract Testing

Test module interface stability:

func TestModuleContract(t *testing.T) {
	t.Parallel()

	// Required inputs
	requiredInputs := []string{"service_name", "environment", "port"}
	
	// Required outputs
	requiredOutputs := []string{"endpoint", "config_file"}

	terraformOptions := &terraform.Options{
		TerraformDir: "../",
	}

	terraform.Init(t, terraformOptions)

	// Verify all required inputs and outputs exist
	for _, input := range requiredInputs {
		description := terraform.GetVariableDescription(t, terraformOptions, input)
		assert.NotEmpty(t, description, "Input %s must be documented", input)
	}
}

Real-World Example: Microservices Platform

Complete modular architecture for microservices.

Project Structure

microservices-platform/
├── modules/
│   ├── foundation/
│   │   └── network/
│   ├── data/
│   │   ├── postgres/
│   │   └── redis/
│   ├── services/
│   │   ├── web-service/
│   │   ├── api-service/
│   │   └── worker-service/
│   └── platform/
│       ├── monitoring/
│       ├── logging/
│       └── tracing/
├── compositions/
│   └── full-stack/
└── environments/
    ├── dev/
    ├── staging/
    └── production/

Foundation Module

modules/foundation/network/main.tf:

variable "environment" {
  type = string
}

variable "cidr_block" {
  type    = string
  default = "10.0.0.0/16"
}

resource "local_file" "network" {
  filename = "${var.environment}-network.yaml"
  content  = yamlencode({
    environment = var.environment
    cidr        = var.cidr_block
    subnets = {
      public  = cidrsubnet(var.cidr_block, 8, 0)
      private = cidrsubnet(var.cidr_block, 8, 1)
      data    = cidrsubnet(var.cidr_block, 8, 2)
    }
  })
}

output "network_config" {
  value = local_file.network.filename
}

output "network_info" {
  value = yamldecode(local_file.network.content)
}

Data Module (PostgreSQL)

modules/data/postgres/main.tf:

variable "environment" {
  type = string
}

variable "database_name" {
  type = string
}

variable "network_info" {
  type = object({
    environment = string
    cidr        = string
    subnets     = map(string)
  })
}

resource "random_password" "db_password" {
  length  = 32
  special = true
}

resource "local_file" "postgres_config" {
  filename = "${var.environment}-${var.database_name}-postgres.yaml"
  content  = yamlencode({
    environment   = var.environment
    database_name = var.database_name
    network       = var.network_info.subnets.data
    host          = "postgres.${var.environment}.internal"
    port          = 5432
    username      = "postgres"
  })
}

resource "local_sensitive_file" "postgres_credentials" {
  filename = "${var.environment}-${var.database_name}-postgres-creds.yaml"
  content  = yamlencode({
    password         = random_password.db_password.result
    connection_string = "postgresql://postgres:${random_password.db_password.result}@postgres.${var.environment}.internal:5432/${var.database_name}"
  })
  
  file_permission = "0600"
}

output "config_file" {
  value = local_file.postgres_config.filename
}

output "database_info" {
  value = {
    host     = "postgres.${var.environment}.internal"
    port     = 5432
    database = var.database_name
    username = "postgres"
  }
}

output "connection_string" {
  value     = "postgresql://postgres:****@postgres.${var.environment}.internal:5432/${var.database_name}"
  sensitive = false
}

Service Module (Web Service)

modules/services/web-service/main.tf:

variable "environment" {
  type = string
}

variable "service_name" {
  type = string
}

variable "port" {
  type    = number
  default = 8080
}

variable "database_info" {
  type = object({
    host     = string
    port     = number
    database = string
    username = string
  })
}

variable "network_info" {
  type = object({
    environment = string
    cidr        = string
    subnets     = map(string)
  })
}

resource "local_file" "service_config" {
  filename = "${var.environment}-${var.service_name}.yaml"
  content  = yamlencode({
    service = {
      name        = var.service_name
      environment = var.environment
      port        = var.port
    }
    network = {
      subnet = var.network_info.subnets.public
    }
    database = var.database_info
  })
}

resource "local_file" "deployment_manifest" {
  filename = "${var.environment}-${var.service_name}-deployment.yaml"
  content  = <<-EOT
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: ${var.service_name}
      namespace: ${var.environment}
    spec:
      replicas: ${var.environment == "production" ? 5 : 2}
      selector:
        matchLabels:
          app: ${var.service_name}
      template:
        metadata:
          labels:
            app: ${var.service_name}
        spec:
          containers:
          - name: ${var.service_name}
            image: ${var.service_name}:latest
            ports:
            - containerPort: ${var.port}
            env:
            - name: PORT
              value: "${var.port}"
            - name: DB_HOST
              value: "${var.database_info.host}"
            - name: DB_PORT
              value: "${var.database_info.port}"
            - name: DB_NAME
              value: "${var.database_info.database}"
  EOT
}

output "service_endpoint" {
  value = "http://${var.service_name}.${var.environment}.internal:${var.port}"
}

output "deployment_manifest" {
  value = local_file.deployment_manifest.filename
}

Platform Module (Monitoring)

modules/platform/monitoring/main.tf:

variable "environment" {
  type = string
}

variable "services" {
  type = map(object({
    endpoint = string
  }))
  description = "Map of services to monitor"
}

resource "local_file" "prometheus_config" {
  filename = "${var.environment}-prometheus.yaml"
  content  = <<-EOT
    global:
      scrape_interval: 15s
      evaluation_interval: 15s
      external_labels:
        environment: ${var.environment}
    
    scrape_configs:
    %{for name, service in var.services~}
    - job_name: '${name}'
      static_configs:
      - targets: ['${replace(service.endpoint, "http://", "")}']
        labels:
          service: '${name}'
          environment: '${var.environment}'
    %{endfor~}
  EOT
}

resource "local_file" "grafana_dashboard" {
  filename = "${var.environment}-grafana-dashboard.json"
  content  = jsonencode({
    dashboard = {
      title = "${var.environment} Services"
      panels = [
        for name, service in var.services : {
          title   = name
          targets = [{ expr = "up{service=\"${name}\"}" }]
        }
      ]
    }
  })
}

output "prometheus_config" {
  value = local_file.prometheus_config.filename
}

output "grafana_dashboard" {
  value = local_file.grafana_dashboard.filename
}

Full Stack Composition

compositions/full-stack/main.tf:

variable "environment" {
  type = string
}

# Foundation
module "network" {
  source = "../../modules/foundation/network"
  
  environment = var.environment
  cidr_block  = var.environment == "production" ? "10.0.0.0/16" : "10.1.0.0/16"
}

# Data layer
module "postgres" {
  source = "../../modules/data/postgres"
  
  environment   = var.environment
  database_name = "app_db"
  network_info  = module.network.network_info
}

module "redis" {
  source = "../../modules/data/redis"
  
  environment  = var.environment
  cache_name   = "app_cache"
  network_info = module.network.network_info
}

# Service layer
module "web_service" {
  source = "../../modules/services/web-service"
  
  environment   = var.environment
  service_name  = "web"
  port          = 8080
  database_info = module.postgres.database_info
  network_info  = module.network.network_info
}

module "api_service" {
  source = "../../modules/services/api-service"
  
  environment   = var.environment
  service_name  = "api"
  port          = 8081
  database_info = module.postgres.database_info
  network_info  = module.network.network_info
}

module "worker_service" {
  source = "../../modules/services/worker-service"
  
  environment   = var.environment
  service_name  = "worker"
  database_info = module.postgres.database_info
  network_info  = module.network.network_info
}

# Platform layer
module "monitoring" {
  source = "../../modules/platform/monitoring"
  
  environment = var.environment
  services = {
    web    = { endpoint = module.web_service.service_endpoint }
    api    = { endpoint = module.api_service.service_endpoint }
    worker = { endpoint = module.worker_service.service_endpoint }
  }
}

output "service_endpoints" {
  value = {
    web    = module.web_service.service_endpoint
    api    = module.api_service.service_endpoint
    worker = module.worker_service.service_endpoint
  }
}

output "monitoring" {
  value = {
    prometheus = module.monitoring.prometheus_config
    grafana    = module.monitoring.grafana_dashboard
  }
}

Environment Configuration

environments/production/main.tf:

terraform {
  required_version = ">= 1.6.0"
  
  backend "s3" {
    bucket = "terraform-state-production"
    key    = "microservices/terraform.tfstate"
    region = "us-east-1"
  }
}

module "platform" {
  source = "../../compositions/full-stack"
  
  environment = "production"
}

output "endpoints" {
  value = module.platform.service_endpoints
}

Usage:

cd environments/production
terraform init
terraform plan
terraform apply

Module Documentation

README Template

modules/service/README.md:

# Service Module

## Description
Deploys a microservice with database connectivity and monitoring.

## Features
- Automatic port assignment
- Database integration
- Health checks
- Deployment manifests
- Environment-specific scaling

## Usage

### Basic
```hcl
module "service" {
  source = "./modules/service"
  
  environment   = "production"
  service_name  = "api"
  database_info = module.database.info
  network_info  = module.network.info
}

Advanced

module "service" {
  source = "./modules/service"
  
  environment  = "production"
  service_name = "api"
  port         = 8080
  
  database_info = {
    host     = "db.internal"
    port     = 5432
    database = "api_db"
    username = "api_user"
  }
  
  network_info = {
    environment = "production"
    cidr        = "10.0.0.0/16"
    subnets = {
      public  = "10.0.1.0/24"
      private = "10.0.2.0/24"
    }
  }
}

Requirements

Name

Version

terraform

>= 1.6.0

random

~> 3.0

Inputs

Name

Description

Type

Default

Required

environment

Environment name

string

n/a

yes

service_name

Service name

string

n/a

yes

port

Service port

number

8080

database_info

Database connection info

object

n/a

yes

network_info

Network configuration

object

n/a

yes

Outputs

Name

Description

service_endpoint

Service HTTP endpoint

deployment_manifest

Path to Kubernetes manifest

Examples

See examples/ directory for:

basic/ - Minimal configuration
complete/ - Full-featured setup
multi-environment/ - Dev, staging, production

Testing

cd test
go test -v -timeout 30m

License

MIT


### Auto-Documentation

```bash
# Install terraform-docs
brew install terraform-docs

# Generate documentation
terraform-docs markdown table . > README.md

# With custom template
terraform-docs markdown table \
  --header-from header.md \
  --footer-from footer.md \
  . > README.md

Module Performance Optimization

Optimization Techniques

1. Minimize dependencies

# Bad: Tight coupling
module "app" {
  database_module = module.database
}

# Good: Loose coupling
module "app" {
  database_endpoint = module.database.endpoint
}

2. Use data sources efficiently

# Cache data source results
data "local_file" "config" {
  filename = "config.yaml"
}

locals {
  config = yamldecode(data.local_file.config.content)
}

# Reuse local.config instead of re-reading

3. Reduce resource count

# Use for_each instead of count for better performance
resource "local_file" "configs" {
  for_each = var.services
  
  filename = "${each.key}.conf"
  content  = each.value.config
}

4. Parallelize independent resources

# These can run in parallel (no dependencies)
module "service_a" { }
module "service_b" { }
module "service_c" { }

Common Module Anti-Patterns

Anti-Pattern 1: God Module

❌ Problem:

module "everything" {
  # 100+ variables
  # 200+ resources
  # Does everything
}

✅ Solution:

module "network" { }
module "database" { }
module "application" { }
module "monitoring" { }

Anti-Pattern 2: Leaky Abstractions

❌ Problem:

module "database" {
  # Exposes internal implementation details
  output "internal_password_hash" { }
  output "backup_script_path" { }
}

✅ Solution:

module "database" {
  # Only expose necessary interface
  output "endpoint" { }
  output "port" { }
}

Anti-Pattern 3: Circular Dependencies

❌ Problem:

module A depends on module B
module B depends on module A

✅ Solution:

Extract shared dependency to module C
module A depends on module C
module B depends on module C

Anti-Pattern 4: Variable Overload

❌ Problem:

variable "config" {
  type = any  # Accepts anything
}

✅ Solution:

variable "config" {
  type = object({
    name    = string
    port    = number
    enabled = bool
  })
}

Advanced Module Techniques

Technique 1: Dynamic Blocks

variable "rules" {
  type = list(object({
    port     = number
    protocol = string
  }))
}

dynamic "rule" {
  for_each = var.rules
  content {
    port     = rule.value.port
    protocol = rule.value.protocol
  }
}

Technique 2: Conditional Resources

resource "local_file" "optional" {
  count = var.create_file ? 1 : 0
  
  filename = "optional.txt"
  content  = "Created conditionally"
}

Technique 3: Module Meta-Arguments

module "service" {
  source = "./modules/service"
  
  # Conditional module
  count = var.enabled ? 1 : 0
  
  # Or for_each
  for_each = var.services
  
  # Dependencies
  depends_on = [module.database]
  
  # Provider selection
  providers = {
    aws = aws.us-west-2
  }
}

Technique 4: Locals for Computation

locals {
  # Complex computations
  services_by_environment = {
    for env, services in var.all_services :
    env => {
      for svc_name, svc_config in services :
      svc_name => merge(svc_config, {
        environment = env
        fqdn        = "${svc_name}.${env}.example.com"
      })
    }
  }
}

What I Learned About Module Architecture

1. Architecture is About Managing Complexity

That 45-minute terraform plan taught me: good architecture makes complexity manageable.

Small, focused modules are easier to understand, test, and change.

2. Composition Beats Inheritance

Building complex systems from simple modules works better than creating complex modules.

Lego blocks, not Swiss Army knives.

3. Clear Interfaces Enable Independence

Well-defined inputs and outputs allow modules to evolve independently.

API contracts prevent breaking changes.

4. Repository Structure Impacts Team Velocity

Mono-repo vs multi-repo isn't just technical—it's organizational.

Choose based on team size and coupling.

5. Versioning Enables Confidence

Pinning module versions prevents surprise breakages.

Semantic versioning communicates change impact.

6. Documentation is Code

Undocumented modules are unusable modules.

terraform-docs makes documentation easy.

7. Testing Complex Modules Requires Strategy

Unit tests for modules, integration tests for compositions.

Contract tests ensure interface stability.

Next Steps

Congratulations! You've mastered advanced module architecture:

✅ Module design principles ✅ Complex module patterns ✅ Composition strategies ✅ Mono-repo vs multi-repo ✅ Module versioning ✅ Registry best practices ✅ Testing complex modules ✅ Performance optimization

Practice Exercises

Exercise 1: Refactor Monolith

1. Take existing monolithic config
2. Identify module boundaries
3. Extract modules
4. Test thoroughly
5. Measure improvement

Exercise 2: Build Composition

1. Create 3-4 simple modules
2. Compose into larger system
3. Version modules
4. Write tests
5. Document usage

Exercise 3: Publish Module

1. Create reusable module
2. Write comprehensive README
3. Add examples
4. Publish to registry
5. Version properly

Coming Up Next

In Article 11: Production Workflows - GitOps, CI/CD, and Deployment Strategies, we'll explore:

GitOps workflows
PR automation
Deployment strategies
Rollback procedures
Disaster recovery
Production best practices

Architecture organizes your code. Workflows deliver it safely to production.

This Week's Challenge: Analyze your current Terraform architecture. Identify opportunities for modularization. Extract at least one reusable module.

See you in Article 11! 🏗️

"That 45-minute terraform plan was a wake-up call. We spent two months refactoring into modules. Now our deploys are 5 minutes and the team moves faster than ever." - Me, architecture convert

PreviousSecurity and Secrets Management NextProduction Workflows and Team Collaboration

Last updated 1 month ago

hashtagTable of Contents

hashtagIntroduction: The Monolith Problem

hashtagWhy Module Architecture Matters

hashtagThe Cost of Poor Architecture

hashtagBenefits of Good Architecture

hashtagArchitecture Evolution

hashtagModule Design Principles

hashtag1. Single Responsibility

hashtag2. Clear Interface

hashtag3. Loose Coupling

hashtag4. High Cohesion

hashtag5. Composition Over Inheritance

hashtagComplex Module Patterns

hashtagPattern 1: Layered Architecture

hashtagPattern 2: Feature Modules

hashtagPattern 3: Environment Wrapper

hashtagPattern 4: Factory Pattern

hashtagPattern 5: Builder Pattern

hashtagModule Composition Strategies

hashtagStrategy 1: Nested Composition

hashtagStrategy 2: Sibling Composition

hashtagStrategy 3: Registry Composition

hashtagStrategy 4: Conditional Composition

hashtagStrategy 5: Data-Driven Composition

hashtagMono-repo vs Multi-repo

hashtagMono-repo Architecture

hashtagMulti-repo Architecture

hashtagHybrid Approach

hashtagDecision Matrix

hashtagModule Versioning Strategies

hashtagSemantic Versioning

hashtagGit Tags

hashtagVersion Constraints

hashtagVersion Pinning

hashtagTerraform Registry Versions

hashtagVersion Update Strategy

hashtagModule Registry Best Practices

hashtagPublishing Modules

hashtagRequirements

hashtagInputs

hashtagOutputs

hashtagExamples

hashtagLicense

hashtagModule Documentation

hashtagTesting Complex Modules

hashtagUnit Testing Modules

hashtagIntegration Testing

hashtagContract Testing

hashtagReal-World Example: Microservices Platform

hashtagProject Structure

hashtagFoundation Module

hashtagData Module (PostgreSQL)

hashtagService Module (Web Service)

hashtagPlatform Module (Monitoring)

hashtagFull Stack Composition

hashtagEnvironment Configuration

hashtagModule Documentation

hashtagREADME Template

hashtagAdvanced

hashtagRequirements

hashtagInputs

hashtagOutputs

hashtagExamples

hashtagTesting

hashtagLicense

hashtagModule Performance Optimization

hashtagOptimization Techniques

hashtagCommon Module Anti-Patterns

hashtagAnti-Pattern 1: God Module

hashtagAnti-Pattern 2: Leaky Abstractions

hashtagAnti-Pattern 3: Circular Dependencies

hashtagAnti-Pattern 4: Variable Overload

hashtagAdvanced Module Techniques

hashtagTechnique 1: Dynamic Blocks

hashtagTechnique 2: Conditional Resources

hashtagTechnique 3: Module Meta-Arguments

hashtagTechnique 4: Locals for Computation

hashtagWhat I Learned About Module Architecture

hashtag1. Architecture is About Managing Complexity

hashtag2. Composition Beats Inheritance

Table of Contents

Introduction: The Monolith Problem

Why Module Architecture Matters

The Cost of Poor Architecture

Benefits of Good Architecture

Architecture Evolution

Module Design Principles

1. Single Responsibility

2. Clear Interface

3. Loose Coupling

4. High Cohesion

5. Composition Over Inheritance

Complex Module Patterns

Pattern 1: Layered Architecture

Pattern 2: Feature Modules

Pattern 3: Environment Wrapper

Pattern 4: Factory Pattern

Pattern 5: Builder Pattern

Module Composition Strategies

Strategy 1: Nested Composition

Strategy 2: Sibling Composition

Strategy 3: Registry Composition

Strategy 4: Conditional Composition

Strategy 5: Data-Driven Composition

Mono-repo vs Multi-repo

Mono-repo Architecture

Multi-repo Architecture

Hybrid Approach

Decision Matrix

Module Versioning Strategies

Semantic Versioning

Git Tags

Version Constraints

Version Pinning

Terraform Registry Versions

Version Update Strategy

Module Registry Best Practices

Publishing Modules

Requirements

Inputs

Outputs

Examples

License

Module Documentation

Testing Complex Modules

Unit Testing Modules

Integration Testing

Contract Testing

Real-World Example: Microservices Platform

Project Structure

Foundation Module

Data Module (PostgreSQL)

Service Module (Web Service)

Platform Module (Monitoring)

Full Stack Composition

Environment Configuration

Module Documentation

README Template

Advanced

Requirements

Inputs

Outputs

Examples

Testing

License

Module Performance Optimization

Optimization Techniques

Common Module Anti-Patterns

Anti-Pattern 1: God Module

Anti-Pattern 2: Leaky Abstractions

Anti-Pattern 3: Circular Dependencies

Anti-Pattern 4: Variable Overload

Advanced Module Techniques

Technique 1: Dynamic Blocks

Technique 2: Conditional Resources

Technique 3: Module Meta-Arguments

Technique 4: Locals for Computation

What I Learned About Module Architecture

1. Architecture is About Managing Complexity

2. Composition Beats Inheritance