Automation Controller Basics

From Command Line Chaos to Centralized Control

For two years, I managed our Ansible automation from the command line. Playbooks lived in Git. Credentials lived in Ansible Vault files. Access control meant "anyone with the Git repo and vault password." Audit trail meant checking Git commit history and hoping people documented what they ran.

Then came the compliance audit.

"Who ran automation against production last Tuesday at 3 PM?" I didn't know.

"Can you prove what changed?" Not easily.

"Who has access to production credentials?" Everyone with the vault password.

That audit failure led me to Automation Controller. Within a week, I had centralized execution, role-based access control, complete audit trails, and secure credential management. The next audit? We passed with flying colors.

This article teaches you the Automation Controller fundamentals I wish I'd learned earlier.

What You'll Learn

• Organizations, Teams, and Users - multi-tenancy structure

• Projects and SCM integration - managing playbook content

• Inventories - static and dynamic host management

• Credentials - secure secret storage and usage

• Job Templates - parameterized automation execution

• Running your first job in Automation Controller

• Activity streams and audit trails

Understanding Automation Controller Structure

Automation Controller organizes resources in a hierarchical structure designed for enterprise multi-tenancy.

Resource Hierarchy

Organizations: Tenancy Boundaries

Organizations are the top-level container for all AAP resources. They provide complete isolation between different teams or business units.

Creating an Organization

Via Web UI:

Access → Organizations → Add
  Name: Engineering
  Description: Engineering team automation
  Max Hosts: 500 (license allocation)
  Instance Groups: default (or specific execution nodes)

Via API:

curl -X POST https://controller.example.com/api/v2/organizations/ \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Engineering",
    "description": "Engineering team automation",
    "max_hosts": 500
  }'

Via awx CLI:

awx organizations create \
  --name "Engineering" \
  --description "Engineering team automation" \
  --max_hosts 500

Real-World Organization Structure

Our production setup:

Organizations:
  - Engineering:
      max_hosts: 500
      teams: [DevOps, Platform, SRE]
      
  - Operations:
      max_hosts: 300
      teams: [Infrastructure, DBAs, Network]
      
  - Security:
      max_hosts: 100
      teams: [Security-Ops, Compliance]

Why separate organizations?

• Complete credential isolation

• Independent resource quotas

• Separate audit trails

• Different team structures

Teams: Role-Based Groupings

Teams group users and assign permissions within an organization.

Creating Teams

# Via awx CLI
awx teams create \
  --name "DevOps" \
  --organization "Engineering" \
  --description "DevOps team members"

awx teams create \
  --name "SRE" \
  --organization "Engineering" \
  --description "Site Reliability Engineers"

Team Permissions

Teams can be granted different permission levels:

Permission Levels:
  - Read: View resources only
  - Use: Execute job templates, use credentials
  - Update: Modify existing resources
  - Admin: Full control including deletion
  
Common Team Roles:
  DevOps Team:
    - Job Templates: Execute + Update
    - Inventories: Use + Read
    - Credentials: Use (but not view secrets)
    
  SRE Team:
    - Job Templates: Execute
    - Inventories: Read
    - Credentials: Use
    
  Admins Team:
    - All Resources: Admin

Adding Users to Teams

# Add user to team
awx teams associate \
  --team "DevOps" \
  --user "[email protected]"

# Grant team permission to job template
awx roles grant \
  --team "DevOps" \
  --type "execute" \
  --job-template "Deploy Web Application"

Projects: SCM Integration

Projects connect Automation Controller to your source code management system (Git, SVN, etc.) where playbooks are stored.

Creating a Project

Via Web UI:

Resources → Projects → Add
  Name: Infrastructure Playbooks
  Organization: Engineering
  SCM Type: Git
  SCM URL: https://gitlab.example.com/ansible/infrastructure.git
  SCM Credential: GitLab-ReadOnly
  SCM Branch: main
  SCM Update Options:
    ✓ Update Revision on Launch
    ✓ Delete on Update
  Cache Timeout: 0 (always update)

Via API:

curl -X POST https://controller.example.com/api/v2/projects/ \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Infrastructure Playbooks",
    "organization": 1,
    "scm_type": "git",
    "scm_url": "https://gitlab.example.com/ansible/infrastructure.git",
    "scm_branch": "main",
    "scm_credential": 5,
    "scm_update_on_launch": true,
    "scm_delete_on_update": true
  }'

Project Best Practices

Use Git Branches for Environments:

Repository Structure:
  branches/main → Production playbooks
  branches/staging → Staging playbooks
  branches/develop → Development playbooks

AAP Projects:
  - Name: "Infra-Production"
    Branch: main
    
  - Name: "Infra-Staging"
    Branch: staging
    
  - Name: "Infra-Development"
    Branch: develop

Enable Update on Launch:

Why: Ensures latest code runs every time
Trade-off: Slight delay before job starts (SCM sync)
Our practice: Enabled for all projects

Use Tags for Releases:

# Tag important releases in Git
git tag -a v1.0.0 -m "Production release 1.0.0"
git push origin v1.0.0

# Create AAP project pointing to tag
SCM Branch/Tag/Commit: refs/tags/v1.0.0

Real-world insight: We maintain separate projects for each branch and environment. This prevents accidental production deployments from development branches.

Inventories: Managing Hosts

Inventories define what hosts automation runs against. AAP supports static and dynamic inventories.

Static Inventory

Manual host definitions:

Via Web UI:

Resources → Inventories → Add → Inventory
  Name: Production Servers
  Organization: Engineering

Hosts → Add
  Name: web01.example.com
  Variables:
    ansible_host: 10.0.1.10
    environment: production
    role: webserver

Groups → Add
  Name: webservers
  Variables:
    http_port: 80
    ssl_enabled: true

Associate Hosts → web01.example.com

Via API:

# Create inventory
curl -X POST https://controller.example.com/api/v2/inventories/ \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Production Servers",
    "organization": 1,
    "variables": "{}"
  }'

# Add host
curl -X POST https://controller.example.com/api/v2/inventories/10/hosts/ \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "web01.example.com",
    "variables": "ansible_host: 10.0.1.10\nenvironment: production"
  }'

Dynamic Inventory

Automatically sync hosts from external sources:

AWS EC2 Example:

Inventory Sources:
  Name: AWS Production
  Source: Amazon EC2
  Credential: AWS-ReadOnly
  Regions: us-east-1, us-west-2
  Instance Filters: tag:Environment=production
  Update on Launch: Yes
  Overwrite: Yes
  Update Cache Timeout: 0

Azure Example:

Inventory Sources:
  Name: Azure Production
  Source: Microsoft Azure Resource Manager
  Credential: Azure-ReadOnly
  Subscription ID: xxx-xxx-xxx
  Resource Groups: production-*
  Tags: Environment:production

Custom Inventory Script:

#!/usr/bin/env python3
# custom_inventory.py

import json

inventory = {
    "webservers": {
        "hosts": ["web01", "web02", "web03"],
        "vars": {
            "http_port": 80
        }
    },
    "databases": {
        "hosts": ["db01", "db02"],
        "vars": {
            "pg_port": 5432
        }
    },
    "_meta": {
        "hostvars": {
            "web01": {"ansible_host": "10.0.1.10"},
            "web02": {"ansible_host": "10.0.1.11"},
            "db01": {"ansible_host": "10.0.2.10"}
        }
    }
}

print(json.dumps(inventory, indent=2))

Real-world inventory strategy:

Our Setup:
  - Static Inventory: On-premises servers (rarely change)
  - AWS EC2 Source: Cloud instances (frequent changes)
  - Azure Source: Cloud instances (frequent changes)
  - Custom Script: Application-specific groupings from CMDB

Credentials: Secure Secret Management

Credentials store authentication information securely, encrypted at rest.

Built-in Credential Types

Common Types:
  - Machine: SSH keys, passwords for Linux/Unix
  - Vault: Ansible Vault passwords
  - Source Control: Git credentials
  - Amazon Web Services: AWS access keys
  - Microsoft Azure Resource Manager: Azure credentials
  - Google Compute Engine: GCP credentials
  - Network: Network device credentials

Creating Credentials

SSH Machine Credential:

Via Web UI:
  Resources → Credentials → Add
    Name: SSH-Production-Servers
    Organization: Engineering
    Credential Type: Machine
    Username: ansible
    SSH Private Key: (paste or upload key)
    Privilege Escalation Method: sudo
    Privilege Escalation Username: root

AWS Credential:

Name: AWS-Production
Credential Type: Amazon Web Services
Access Key: AKIAIOSFODNN7EXAMPLE
Secret Key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Vault Credential:

Name: Ansible-Vault-Password
Credential Type: Vault
Vault Password: (your vault password)
Vault Identifier: prod (for multi-vault)

Credential Security

Key features:

Encryption:
  - Encrypted in database with SECRET_KEY
  - Never displayed in UI after creation
  - Not accessible via API GET requests
  
Access Control:
  - Users can USE credentials without viewing them
  - RBAC controls who can create/modify
  - Audit trail tracks credential usage
  
Integration:
  - HashiCorp Vault lookup
  - CyberArk integration
  - Azure Key Vault

Real-world practice:

Our Credential Strategy:
  - SSH Keys: Rotated every 90 days
  - Cloud Credentials: Synced from HashiCorp Vault
  - Vault Passwords: Different per environment
  - Network Credentials: Per device type
  
Access Pattern:
  - Developers: CAN use credentials
  - Developers: CANNOT view credentials
  - Security team: CAN view usage logs
  - Only admins: CAN modify credentials

Job Templates: Parameterized Execution

Job Templates combine Projects, Inventories, and Credentials into executable automation.

Creating a Job Template

Via Web UI:
  Resources → Templates → Add → Job Template
    Name: Deploy Web Application
    Job Type: Run
    Inventory: Production Servers
    Project: Infrastructure Playbooks
    Playbook: deploy_webapp.yml
    Credentials:
      - SSH-Production-Servers
      - AWS-Production (if using AWS modules)
      - Ansible-Vault-Password
    Variables:
      app_version: "1.0.0"
      deploy_region: "us-east-1"
    Options:
      ✓ Enable Privilege Escalation
      ✓ Enable Fact Storage
      ✓ Allow Simultaneous (if safe)
    Limits: webservers (optional, can override at launch)
    Verbosity: 0 (Normal)
    Job Tags: deploy,configure (optional)
    Skip Tags: (optional)
    Timeout: 0 (no timeout)

Advanced Job Template Features

Survey (User Input):

Add Survey:
  Prompt: Application Version
  Variable Name: app_version
  Type: Text
  Default: latest
  Required: Yes
  
  Prompt: Target Environment
  Variable Name: target_env
  Type: Multiple Choice
  Choices: [dev, staging, production]
  Required: Yes

Prompt on Launch:

Enable:
  ✓ Prompt for Inventory
  ✓ Prompt for Credentials
  ✓ Prompt for Variables
  ✓ Prompt for Limit
  ✓ Prompt for Tags

Instance Groups:

# Run on specific execution nodes
Instance Groups: [aws-executors, on-prem-executors]

Why: Control WHERE playbooks execute
Use case: Run AWS playbooks on AWS execution nodes for better performance

Running Your First Job

Via Web UI

1. Navigate to Templates
2. Click rocket icon next to job template
3. Fill in survey prompts (if any)
4. Review and confirm
5. Watch real-time output

Via API

# Launch job
curl -X POST \
  https://controller.example.com/api/v2/job_templates/42/launch/ \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "extra_vars": {
      "app_version": "1.2.0",
      "target_env": "production"
    },
    "limit": "webservers"
  }'

# Response includes job ID
{
  "job": 12345,
  "url": "/api/v2/jobs/12345/"
}

# Monitor job status
curl https://controller.example.com/api/v2/jobs/12345/ \
  -H "Authorization: Bearer $TOKEN"

Via awx CLI

# Launch job template
awx job_templates launch "Deploy Web Application" \
  --extra_vars '{"app_version": "1.2.0"}' \
  --limit webservers \
  --monitor

# List running jobs
awx jobs list --status running

# Get job output
awx jobs stdout 12345

Job Execution Flow

Activity Stream and Audit Trail

Every action in Automation Controller is logged in the Activity Stream.

Viewing Activity

Via Web UI:

Views → Activity Stream
  Filters:
    - Action: Create, Update, Delete, Associate
    - Object Type: Job Template, User, Credential
    - Initiator: Specific user
    - Date Range: Last 7 days

What's Logged

Logged Actions:
  - Resource creation/modification/deletion
  - User logins and logouts
  - Permission changes
  - Job launches and completions
  - Credential usage (but not values)
  - Configuration changes
  
Each Log Entry Contains:
  - Timestamp
  - User (who)
  - Action (what)
  - Object (affected resource)
  - Changes (what changed)
  - IP Address (from where)

Real-World Audit Example

{
  "id": 54321,
  "type": "activity_stream",
  "timestamp": "2026-01-05T14:30:00Z",
  "operation": "update",
  "object1": "job_template",
  "object1_name": "Deploy Web Application",
  "actor": "[email protected]",
  "changes": {
    "variables": {
      "old": "app_version: 1.0.0",
      "new": "app_version: 1.2.0"
    }
  }
}

Use cases:

• Compliance audits: "Who modified this job template?"

• Security incidents: "Who accessed production last night?"

• Troubleshooting: "What changed before automation broke?"

Practical Example: Complete Setup

Let's walk through setting up a complete automation workflow.

Scenario: Web Application Deployment

Step 1: Create Organization

awx organizations create --name "WebTeam"

Step 2: Create Team

awx teams create --name "Developers" --organization "WebTeam"

Step 3: Add Users

awx users create --username "alice" --email "[email protected]" --password "SecurePass123"
awx teams associate --team "Developers" --user "alice"

Step 4: Create Credentials

# SSH Credential
awx credentials create \
  --name "WebServers-SSH" \
  --credential_type "Machine" \
  --organization "WebTeam" \
  --inputs '{"username": "deploy", "ssh_key_data": "@~/.ssh/deploy_rsa"}'

# Git Credential
awx credentials create \
  --name "GitLab-ReadOnly" \
  --credential_type "Source Control" \
  --organization "WebTeam" \
  --inputs '{"username": "git-readonly", "password": "token123"}'

Step 5: Create Project

awx projects create \
  --name "WebApp-Playbooks" \
  --organization "WebTeam" \
  --scm_type git \
  --scm_url "https://gitlab.example.com/webteam/playbooks.git" \
  --scm_credential "GitLab-ReadOnly" \
  --scm_branch "main" \
  --scm_update_on_launch true

Step 6: Create Inventory

awx inventory create \
  --name "WebServers" \
  --organization "WebTeam"

awx hosts create \
  --name "web01.example.com" \
  --inventory "WebServers" \
  --variables '{"ansible_host": "10.0.1.10"}'

Step 7: Create Job Template

awx job_templates create \
  --name "Deploy WebApp" \
  --job_type run \
  --inventory "WebServers" \
  --project "WebApp-Playbooks" \
  --playbook "deploy.yml" \
  --credentials "WebServers-SSH" \
  --extra_vars '{"app_version": "latest"}' \
  --ask_variables_on_launch true

Step 8: Grant Permissions

awx roles grant \
  --team "Developers" \
  --type execute \
  --job-template "Deploy WebApp"

Step 9: Launch Job

awx job_templates launch "Deploy WebApp" \
  --extra_vars '{"app_version": "v1.2.0"}' \
  --monitor

Key Takeaways

✅ Organizations provide multi-tenancy isolation ✅ Teams group users with role-based permissions ✅ Projects connect to Git for version-controlled playbooks ✅ Inventories can be static or dynamically synced ✅ Credentials are encrypted and access-controlled ✅ Job Templates parameterize automation execution ✅ Activity Stream provides complete audit trail

What's Next

Now that you understand Automation Controller basics, the next article dives deeper into Projects and Inventories - multi-repository management, cloud dynamic inventories, smart inventories, and advanced inventory patterns.

---

Next Article: Projects and Inventories in AAP →

Additional Resources

• Automation Controller User Guide

• AWX CLI Documentation

• Job Template Best Practices

---

Part of the Ansible Automation Platform 101 Series