Integrating TinyLlama with Ansible: Building an LLM-Powered Automation Framework

June 21, 2025

The Vision: AI-Powered Infrastructure Automation

When I first started this project, I had a simple but ambitious goal: merge the power of modern language models with infrastructure automation tools. Ansible has long been my go-to tool for infrastructure management, but I wanted to make it more accessible and powerful through natural language interfaces.

Enter TinyLlama - a compact yet capable language model that can run locally without requiring massive GPU resources. The beauty of this approach? All the AI smarts stay on your local machine or server - no need to send your infrastructure details to external APIs. This local-first design makes it perfect for analyzing and generating Ansible playbooks even in secure environments or air-gapped systems.

Project Overview: The Ansible-TinyLlama Integration

The project combines Ansible's powerful automation capabilities with TinyLlama's natural language understanding to create a framework that can:

• Convert natural language descriptions into Ansible playbooks

• Analyze existing playbooks for improvements and best practices

• Provide AI-powered recommendations when playbooks fail

• Enable intelligent automation across both Linux and Windows environments

But beyond these features, I wanted to create a developer-friendly environment that would make testing and developing these capabilities straightforward.

The Architecture

The system is built with a clear separation of concerns:

This modular design allows me to improve each component independently while maintaining clean interfaces between them.

The Workflow: How It Actually Works

One of the most interesting features is the playbook analysis capability. Here's the sequence of how it processes an existing Ansible playbook:

The beauty of this workflow is that it can analyze complex playbooks and provide meaningful feedback without requiring deep understanding of Ansible's internals. The analysis covers multiple aspects:

1. Overview: A summary of what the playbook does

2. Potential Issues: Bugs, errors, or problematic patterns

3. Security Concerns: Potential security vulnerabilities

4. Best Practice Recommendations: Ways to improve and optimize the playbook

Here's another key workflow - playbook generation from natural language:

This generation capability transforms how teams can create infrastructure automation. Instead of starting from scratch, users can describe what they want in plain English, and the system generates the appropriate Ansible code.

Testing with Mock Hosts: A Cross-Platform Approach

One of the most powerful aspects of this project is the comprehensive testing environment featuring both Linux and Windows mock hosts. This enables thorough testing of playbooks against different operating systems without requiring actual infrastructure.

The Mock Host Architecture

Testing on Mock Windows Host

The Windows mock host uses PowerShell Core in a Docker container to simulate a Windows environment. Here's how to test against it:

# Start the mock Windows host
./dev.sh test-windows

# The test-windows command performs these steps:
# 1. Starts the mock_windows_host container
# 2. Sets up proper SSH known_hosts
# 3. Runs a simple verification test
# 4. Offers to run full PowerShell test if requested

The mock Windows host provides:

• SSH access with the username 'ansible_user'

• PowerShell Core 7.x

• Windows-like directory structures

• Support for testing PowerShell scripts via Ansible's raw module

Here's a sample playbook specifically designed for Windows over SSH:

---
- name: Test PowerShell Execution on Windows
  hosts: windows
  gather_facts: false
  
  tasks:
    - name: Execute PowerShell command
      ansible.builtin.raw: /opt/microsoft/powershell/7/pwsh -Command "Get-Process | Select-Object -First 5"
      register: raw_output
    
    - name: Display command output
      ansible.builtin.debug:
        var: raw_output.stdout_lines

Testing on Mock Linux Host

The Linux mock host uses an Ubuntu 22.04 container with SSH access:

# Start the mock Linux host
./dev.sh test-linux

# The test-linux command:
# 1. Starts the mock_linux_host container
# 2. Sets up SSH known_hosts
# 3. Runs a simple connection test
# 4. Can run full test playbooks when requested

The mock Linux host provides:

• SSH access with the username 'ansible_user'

• Python 3 pre-installed for Ansible compatibility

• Support for both password and key-based authentication

• Sudo privileges for the ansible_user

Working with Playbook Analysis and Generation

The project offers two core capabilities that are currently in active development: playbook analysis and playbook generation. Here's how to use them and how they work behind the scenes.

Analyzing Existing Playbooks

Playbook analysis takes an existing Ansible playbook and provides insights, improvements, and best practice recommendations. To use this feature:

# Analyze a playbook using the CLI
./dev.sh analyze-playbook path/to/your/playbook.yml

# Analyze with different model options
./dev.sh analyze-playbook path/to/complex/playbook.yml tiny

Behind the scenes, the analysis process:

1. Parses the YAML file and validates it's a proper Ansible playbook

2. Creates a specialized prompt for the LLM that includes playbook analysis instructions

3. Feeds the prompt and playbook to the TinyLlama model

4. Structures the response into sections (summary, issues, security concerns, best practices)

5. Formats and presents the results

The analysis output provides actionable insights like:

Playbook Analysis Results:

Summary:
This playbook installs and configures Nginx on web servers with proper 
SSL configuration and basic security hardening.

Potential Issues:
• No handlers to restart Nginx after configuration changes
• Missing idempotency checks in some tasks
• No verification that the SSL certificates exist before configuring SSL

Security Considerations:
• Using default SSL protocols without specifying secure versions
• Not setting secure TLS cipher suites explicitly
• No configuration for HTTP security headers

Best Practice Recommendations:
• Add a handler to restart Nginx only when needed
• Include tags for selective task execution
• Verify certificate existence before enabling SSL
• Add proper error handling for certificate issues

Generating Playbooks from Natural Language

The playbook generation feature transforms natural language descriptions into working Ansible playbooks:

# Generate a playbook from description
python -m src.main cli generate-playbook "Install and configure Nginx with rate limiting on all web servers"

# Save to file
python -m src.main cli generate-playbook "Setup PostgreSQL database with user authentication" -o setup_postgres.yml

The generation process works by:

1. Taking the natural language description and creating a specialized prompt

2. Adding context about Ansible best practices and task structure

3. Having the model generate YAML in the proper Ansible format

4. Validating the generated YAML is both valid YAML and follows Ansible structure

5. Optional post-processing to add necessary comments and documentation

Here's an example of a generated playbook from the description "Install and configure Nginx with rate limiting":

---
- name: Install and configure Nginx with rate limiting
  hosts: web_servers
  become: true
  vars:
    nginx_rate_limit: 10r/s
    nginx_rate_limit_zone: "$binary_remote_addr"
    nginx_rate_limit_size: 10m

  tasks:
    - name: Install Nginx package
      apt:
        name: nginx
        state: present
        update_cache: yes
      when: ansible_os_family == "Debian"

    - name: Install Nginx package (RedHat/CentOS)
      yum:
        name: nginx
        state: present
      when: ansible_os_family == "RedHat"

    - name: Ensure Nginx service is enabled and started
      service:
        name: nginx
        state: started
        enabled: yes

    - name: Configure rate limiting in Nginx
      blockinfile:
        path: /etc/nginx/nginx.conf
        insertbefore: "http {"
        block: |
          # Rate limiting configuration
          limit_req_zone {{ nginx_rate_limit_zone }} zone=mylimit:{{ nginx_rate_limit_size }} rate={{ nginx_rate_limit }};

    - name: Apply rate limiting to server blocks
      blockinfile:
        path: /etc/nginx/sites-available/default
        insertafter: "location / {"
        block: |
          # Enable rate limiting
          limit_req zone=mylimit burst=20 nodelay;

    - name: Reload Nginx to apply changes
      service:
        name: nginx
        state: reloaded

Integration Testing the Generated Playbooks

One of the key strengths of this project is the ability to immediately test generated playbooks on the mock hosts:

# Generate a playbook
python -m src.main cli generate-playbook "Install and configure Nginx with rate limiting" -o nginx_playbook.yml

# Test it on the mock Linux host
docker compose -f docker-compose.dev.yml exec test_runner ansible-playbook -i tests/mock_linux_host/inventory.ini nginx_playbook.yml -v

This testing process verifies that:

1. The generated YAML is valid and properly formatted

2. The tasks run correctly on the target host

3. The playbook achieves the intended outcome

4. Resources are properly configured

For Windows playbooks, similar testing can be done:

# Generate a Windows-specific playbook
python -m src.main cli generate-playbook "Create Windows scheduled task to run script daily" -o windows_task.yml

# Test on the mock Windows host
docker compose -f docker-compose.dev.yml exec test_runner ansible-playbook -i tests/mock_windows_host/inventory.ini windows_task.yml -v

Development Workflow for Analyzing and Generating Playbooks

The typical development workflow for working with playbook analysis and generation looks like this:

This iterative approach combines the strengths of both the generation and analysis capabilities, creating a feedback loop that continuously improves playbook quality.

Lessons Learned

Through developing and testing the analysis and generation features, I've learned several key lessons:

1. Specific Prompts Matter: The quality of playbook generation greatly depends on how specific and clear the prompt is. Including details about idempotency, error handling, and target environment significantly improves the results.

2. Cross-Platform Differences: Generated playbooks often need tweaks to work properly across different platforms. Windows playbooks in particular require special attention due to the use of PowerShell instead of shell commands.

3. Best Practices Need Reinforcement: The analysis feature works best when it has a comprehensive understanding of Ansible best practices embedded in its prompt templates.

4. Testing Environment is Crucial: Having realistic mock hosts allows for immediate verification of generated playbooks, dramatically accelerating the development cycle.

5. Target Platform Detection: Automatically detecting the target platform (Windows vs Linux) from the natural language description improves generation accuracy significantly.

Future Improvements for Playbook Analysis and Generation

I'm continuing to improve the playbook analysis and generation features with several exciting enhancements on the roadmap:

1. Interactive Playbook Generation: Creating an interactive mode that guides users through generating complex playbooks step-by-step, asking clarifying questions along the way:

   System: What kind of servers will this playbook target?
   User: Web servers running Nginx
   System: Should I include SSL configuration?
   User: Yes, with modern TLS settings
   System: Generating secure Nginx configuration with modern TLS...

2. Execution Feedback Loop: Implementing a system where playbook execution results (successful or failed) are fed back to the LLM to improve future generations:

3. Role-Based Generation: Extending the generation capability to create full Ansible roles with proper directory structure, defaults, tasks, handlers, and templates:

   User: Generate a complete Nginx role with SSL support
   System: Creating role structure with:
   - tasks/main.yml - Core installation tasks
   - tasks/ssl.yml - SSL configuration
   - templates/nginx.conf.j2 - Template with variables
   - defaults/main.yml - Default variables
   - handlers/main.yml - Restart handlers
   - ...

4. Multi-Platform Awareness: Enhancing the analyzer to identify platform-specific issues and ensure playbooks work consistently across Linux distributions and Windows versions:

   Platform-specific issues:
   • This apt task will fail on RedHat systems
   • The file path uses Windows format but targets Linux hosts
   • This PowerShell command requires PS 5.0 or higher

5. Security Analysis Enhancement: Adding specialized security scanning to identify potential vulnerabilities or misconfigurations:

   Security Findings:
   • Using unencrypted HTTP protocol for package downloads
   • World-readable permissions on sensitive configuration files
   • SSH configuration allows root login and password authentication
   • No timeout settings for idle sessions

6. Variable Optimization: Analyzing playbooks for hardcoded values that should be variables and suggesting improvements:

   Variable Recommendations:
   • Convert hardcoded port "8080" to variable
   • Extract repeated domain name "example.com" as variable
   • Create map variable for package names by OS family

7. Inventory-Aware Analysis: Extending the analyzer to consider actual inventory data when analyzing playbooks:

   Inventory Considerations:
   • This playbook targets 'db_servers' but your inventory has none
   • Server 'app01' is using Ubuntu but this task only supports CentOS
   • No Windows hosts found for this Windows-specific playbook

The Processing Flow

The data flow through the system follows this sequence:

Error Handling

The system has robust error handling mechanisms:

Try It Yourself!

If you're interested in trying out this project:

1. Check out the GitHub repository

2. Follow the installation instructions in the README.md

3. Use the dev.sh setup command to get started quickly

4. Try analyzing a playbook with ./dev.sh analyze-playbook your_playbook.yml

Your feedback and contributions are always welcome!

---

This blog post reflects my personal experiences building and working with the Ansible-TinyLlama Integration project. The code examples and diagrams are simplified versions of the actual implementation. Feel free to reach out with questions or suggestions!