My Journey with the CALMS Model in DevOps

Published: July 1, 2025

When I first encountered the CALMS framework five years ago, I was struggling with what many organizations face: the gap between development and operations teams seemed insurmountable. Despite having all the right tools and processes in place, our DevOps transformation felt more like a buzzword exercise than a meaningful cultural shift. That's when my manager introduced me to CALMS—a framework that would fundamentally change how I approached DevOps implementation.

CALMS, coined by Jez Humble in "The DevOps Handbook," stands for Culture, Automation, Lean, Measurement, and Sharing. But more than just an acronym, it became my roadmap for assessing and improving our DevOps maturity. In this post, I'll share my personal journey implementing CALMS, with practical examples using Jira and GitLab that helped transform our team dynamics and delivery capabilities.

Understanding CALMS: More Than Just Another Framework

Before diving into my implementation story, let me explain why CALMS resonated with me. Unlike other frameworks that focus primarily on tools and processes, CALMS puts human collaboration at its center. It recognizes that DevOps isn't just about automating deployments—it's about creating a culture where development, operations, and business teams work as one cohesive unit.

Here's a sequence diagram showing how CALMS components interact in a typical DevOps workflow:

The "C" in CALMS: Building a Collaborative Culture

The Challenge: Silos Everywhere

When I joined my current organization, the development and operations teams operated in complete isolation. Developers would "throw code over the wall" to operations, who would then struggle with deployment issues. Blame was common, collaboration was rare, and our incident response resembled a game of hot potato.

The Cultural Transformation

Implementing the culture component of CALMS wasn't about mandating collaboration—it was about creating systems and practices that naturally encouraged it. Here's how we used Jira and GitLab to break down silos:

Shared Jira Project Structure: Instead of separate projects for Dev and Ops, we created unified project structures that promoted visibility and shared ownership:

# Example Jira Project Structure
Project: "Product X Development & Operations"

Epic Types:
  - Feature Development
  - Infrastructure Improvement
  - Incident Response
  - Performance Optimization

Issue Types:
  - Story (Development)
  - Task (Operations)
  - Bug (Shared ownership)
  - Incident (Cross-team collaboration)

Custom Fields:
  - Deployment Environment
  - Performance Impact
  - Security Considerations
  - Operations Review Required (Boolean)

Cross-Functional Jira Workflows: We designed workflows that required both development and operations input:

# Example Bug Resolution Workflow
States:
  - Open
  - In Development
  - Ops Review Required
  - Ready for Testing
  - In Testing
  - Ready for Deploy
  - Deployed
  - Verification Complete
  - Closed

Transitions:
  "Ops Review Required" → "Ready for Testing"
    Conditions: Operations team approval required
    Validators: Operations team must add deployment notes

GitLab Integration for Transparency: We configured GitLab to automatically update Jira issues, creating a shared narrative of changes:

# .gitlab-ci.yml example showing Jira integration
stages:
  - test
  - build
  - deploy
  - notify

variables:
  JIRA_PROJECT_KEY: "PRODX"

test_job:
  stage: test
  script:
    - run_tests.sh
  after_script:
    - |
      if [ "$CI_JOB_STATUS" == "success" ]; then
        curl -X POST -H "Content-Type: application/json" \
        -d "{\"body\": \"✅ Tests passed for commit ${CI_COMMIT_SHA} - Ready for build\"}" \
        "https://jira.company.com/rest/api/2/issue/${CI_COMMIT_REF_NAME}/comment"
      fi

deploy_job:
  stage: deploy
  script:
    - deploy_to_staging.sh
  after_script:
    - |
      curl -X POST -H "Content-Type: application/json" \
      -d "{\"body\": \"🚀 Deployed to staging: ${CI_ENVIRONMENT_URL}\"}" \
      "https://jira.company.com/rest/api/2/issue/${CI_COMMIT_REF_NAME}/comment"

Results of Cultural Change

The impact was dramatic. Within six months:

  • Our incident response time decreased by 60%

  • Cross-team knowledge sharing increased measurably (tracked through Confluence page views and Slack interactions)

  • Employee satisfaction scores improved across both teams

  • We eliminated the "blame game" mentality entirely

The "A" in CALMS: Automation That Empowers Teams

Beyond CI/CD: Automation for Collaboration

While most teams focus on deployment automation, I discovered that the most impactful automation often involves the mundane tasks that create friction between teams.

Automated Jira Issue Management: We created GitLab automation that managed Jira issues throughout the development lifecycle:

# GitLab CI job for automatic Jira transitions
update_jira:
  stage: notify
  script:
    - |
      # Extract Jira issue key from branch name or commit message
      JIRA_KEY=$(echo $CI_COMMIT_REF_NAME | grep -o '[A-Z]\+-[0-9]\+' | head -1)
      
      if [ ! -z "$JIRA_KEY" ]; then
        case "$CI_JOB_STAGE" in
          "test")
            TRANSITION_ID="31" # Move to "In Testing"
            ;;
          "deploy")
            TRANSITION_ID="41" # Move to "Deployed"
            ;;
        esac
        
        curl -X POST \
        -H "Content-Type: application/json" \
        -d "{\"transition\":{\"id\":\"$TRANSITION_ID\"}}" \
        "https://jira.company.com/rest/api/2/issue/$JIRA_KEY/transitions"
      fi

Automated Environment Provisioning: Using GitLab's Auto DevOps capabilities, we automated environment creation tied to Jira epic lifecycles:

# .gitlab-ci.yml for dynamic environment creation
create_environment:
  stage: deploy
  script:
    - |
      # Create environment based on Jira epic
      EPIC_KEY=$(curl -s "https://jira.company.com/rest/api/2/issue/$JIRA_KEY" | jq -r '.fields.customfield_10014')
      ENVIRONMENT_NAME=$(echo "$EPIC_KEY" | tr '[:upper:]' '[:lower:]')
      
      # Deploy using Helm with dynamic values
      helm upgrade --install $ENVIRONMENT_NAME ./chart \
        --set image.tag=$CI_COMMIT_SHA \
        --set ingress.host=$ENVIRONMENT_NAME.staging.company.com \
        --set jira.epic=$EPIC_KEY
  environment:
    name: staging/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG.staging.company.com
    on_stop: cleanup_environment

Automation Success Metrics

Our automation efforts yielded quantifiable benefits:

  • Deployment frequency increased from weekly to multiple times per day

  • Manual environment setup time reduced from 4 hours to 15 minutes

  • Configuration drift incidents dropped to zero

  • Developer productivity increased by 40% (measured by story points delivered per sprint)

The "L" in CALMS: Lean Thinking and Continuous Improvement

Identifying Waste in Our Processes

The Lean component of CALMS taught me to view our development process through the lens of value stream mapping. Using data from both Jira and GitLab, we identified several types of waste:

Waiting Time Analysis:

-- Example Jira JQL query to identify bottlenecks
project = "PRODX" AND 
status WAS "Waiting for Deployment" FOR MORE THAN "2d" 
ORDER BY created DESC

Work-in-Progress (WIP) Limits: We implemented WIP limits in our Jira board and enforced them through GitLab merge request policies:

# .gitlab-ci.yml job to check WIP limits
check_wip_limits:
  stage: validate
  script:
    - |
      # Get current WIP count from Jira
      IN_PROGRESS=$(curl -s "https://jira.company.com/rest/api/2/search?jql=project=PRODX AND status IN ('In Progress', 'Code Review')" | jq '.total')
      
      if [ $IN_PROGRESS -gt 5 ]; then
        echo "❌ WIP limit exceeded: $IN_PROGRESS items in progress (limit: 5)"
        echo "Please complete existing work before starting new items"
        exit 1
      fi
      
      echo "✅ WIP limit check passed: $IN_PROGRESS/5 items in progress"
  rules:
    - if: $CI_MERGE_REQUEST_ID

Continuous Improvement Through Retrospectives

We automated retrospective data collection using GitLab and Jira APIs:

# Python script to generate retrospective data
import requests
import json
from datetime import datetime, timedelta

def generate_retrospective_report():
    """Generate automated retrospective data from GitLab and Jira."""
    
    # Last sprint data
    end_date = datetime.now()
    start_date = end_date - timedelta(days=14)  # 2-week sprint
    
    # GitLab metrics
    gitlab_data = {
        'merge_requests': get_gitlab_mrs(start_date, end_date),
        'deployments': get_gitlab_deployments(start_date, end_date),
        'pipeline_failures': get_pipeline_failures(start_date, end_date)
    }
    
    # Jira metrics
    jira_data = {
        'completed_stories': get_completed_stories(start_date, end_date),
        'cycle_time': calculate_cycle_time(start_date, end_date),
        'escaped_bugs': get_escaped_bugs(start_date, end_date)
    }
    
    # Generate insights
    insights = generate_insights(gitlab_data, jira_data)
    
    # Post to Confluence for team review
    post_retrospective_data(insights)
    
    return insights

def generate_insights(gitlab_data, jira_data):
    """Generate actionable insights from the data."""
    insights = []
    
    # Pipeline failure analysis
    if gitlab_data['pipeline_failures'] > 3:
        insights.append({
            'type': 'concern',
            'title': 'High Pipeline Failure Rate',
            'description': f"We had {gitlab_data['pipeline_failures']} pipeline failures this sprint",
            'action': 'Review test stability and infrastructure reliability'
        })
    
    # Cycle time analysis
    avg_cycle_time = jira_data['cycle_time']['average']
    if avg_cycle_time > 5:  # days
        insights.append({
            'type': 'concern',
            'title': 'Long Cycle Time',
            'description': f"Average cycle time was {avg_cycle_time} days",
            'action': 'Identify bottlenecks in development workflow'
        })
    
    return insights

Lean Implementation Results

Our lean approach delivered substantial improvements:

  • Lead time reduced from 2 weeks to 4 days

  • Deployment success rate increased from 75% to 98%

  • Feature feedback loop shortened from 1 month to 1 week

  • Team satisfaction with process improved by 50%

The "M" in CALMS: Measurement-Driven Decision Making

Building a Measurement Culture

The measurement component transformed how we made decisions. Instead of relying on gut feelings, we built comprehensive dashboards using data from Jira, GitLab, and our monitoring systems.

GitLab Analytics Integration:

# .gitlab-ci.yml job to collect deployment metrics
collect_metrics:
  stage: post-deploy
  script:
    - |
      # Collect deployment metrics
      DEPLOY_TIME=$(date -Iseconds)
      COMMIT_COUNT=$(git rev-list --count HEAD^..HEAD)
      
      # Send metrics to monitoring system
      curl -X POST "https://monitoring.company.com/api/v1/metrics" \
      -H "Content-Type: application/json" \
      -d "{
        \"timestamp\": \"$DEPLOY_TIME\",
        \"deployment\": {
          \"project\": \"$CI_PROJECT_NAME\",
          \"environment\": \"$CI_ENVIRONMENT_NAME\",
          \"commit_sha\": \"$CI_COMMIT_SHA\",
          \"pipeline_duration\": $CI_PIPELINE_DURATION,
          \"commit_count\": $COMMIT_COUNT
        }
      }"
      
      # Update Jira with deployment metrics
      JIRA_KEY=$(echo $CI_COMMIT_REF_NAME | grep -o '[A-Z]\+-[0-9]\+')
      if [ ! -z "$JIRA_KEY" ]; then
        curl -X PUT "https://jira.company.com/rest/api/2/issue/$JIRA_KEY" \
        -H "Content-Type: application/json" \
        -d "{
          \"fields\": {
            \"customfield_10100\": \"$DEPLOY_TIME\",
            \"customfield_10101\": $CI_PIPELINE_DURATION
          }
        }"
      fi

Key Metrics Dashboard: We created a comprehensive dashboard tracking:

  1. DORA Metrics:

    • Deployment Frequency: 15.2 deployments per day (up from 0.5 per week)

    • Lead Time for Changes: 4.1 days (down from 14 days)

    • Change Failure Rate: 2.1% (down from 25%)

    • Mean Time to Recovery: 47 minutes (down from 8 hours)

  2. Team Health Metrics:

    • Sprint Velocity: Consistent 35-40 story points

    • Cycle Time: 4.2 days average

    • Work-in-Progress: Maintained under 5 items

    • Team Satisfaction: 8.2/10 (up from 5.1/10)

  3. Business Impact Metrics:

    • Feature Time-to-Market: 1.5 weeks (down from 2 months)

    • Customer Issue Resolution: 1.2 days (down from 1 week)

    • System Uptime: 99.8% (up from 95.2%)

Measurement Tools Integration

Custom Jira Dashboard:

// Gadget configuration for CALMS metrics dashboard
{
  "gadgets": [
    {
      "title": "Deployment Frequency",
      "type": "custom-chart",
      "jql": "project = PRODX AND status = Deployed",
      "groupBy": "resolved",
      "timeframe": "last30days"
    },
    {
      "title": "Cycle Time Analysis",
      "type": "control-chart",
      "jql": "project = PRODX AND resolved >= -30d",
      "metrics": ["cycle_time", "lead_time"]
    },
    {
      "title": "Cross-Team Collaboration",
      "type": "collaboration-index",
      "teams": ["Development", "Operations", "QA"],
      "metrics": ["shared_issues", "cross_team_comments"]
    }
  ]
}

The "S" in CALMS: Sharing Knowledge and Responsibility

Breaking Down Knowledge Silos

The sharing component became perhaps the most transformative aspect of our CALMS implementation. We moved from hoarded knowledge to collaborative intelligence.

Automated Documentation Generation:

# GitLab CI job to generate and share documentation
update_documentation:
  stage: document
  script:
    - |
      # Generate API documentation
      swagger-codegen generate -i api/swagger.yaml -l html2 -o docs/api/
      
      # Generate deployment runbook
      envsubst < templates/runbook.md.template > docs/runbooks/deployment.md
      
      # Update Confluence space
      python scripts/update_confluence.py \
        --space "TECH" \
        --page "Product X Documentation" \
        --content docs/
      
      # Update Jira epic with documentation links
      EPIC_KEY=$(curl -s "https://jira.company.com/rest/api/2/issue/$JIRA_KEY" | jq -r '.fields.customfield_10014')
      curl -X PUT "https://jira.company.com/rest/api/2/issue/$EPIC_KEY" \
      -H "Content-Type: application/json" \
      -d "{
        \"fields\": {
          \"customfield_10200\": \"https://confluence.company.com/display/TECH/Product+X+Documentation\"
        }
      }"
  artifacts:
    paths:
      - docs/
    expire_in: 1 week

Shared On-Call Responsibilities: We implemented a rotation system where developers participated in on-call duties:

# PagerDuty integration script
import requests
from datetime import datetime, timedelta

def create_shared_oncall_schedule():
    """Create on-call schedule involving both dev and ops."""
    
    # Get team members from Jira
    dev_team = get_jira_team_members("Development")
    ops_team = get_jira_team_members("Operations")
    
    # Create mixed rotation
    mixed_rotation = []
    for i in range(len(dev_team)):
        mixed_rotation.extend([
            {"type": "dev", "member": dev_team[i]},
            {"type": "ops", "member": ops_team[i % len(ops_team)]}
        ])
    
    # Update PagerDuty schedule
    update_pagerduty_schedule(mixed_rotation)
    
    # Update Jira dashboard with current on-call
    update_jira_oncall_dashboard(mixed_rotation[0])

def handle_incident_response(incident_id):
    """Automated incident response with cross-team collaboration."""
    
    # Create Jira incident ticket
    jira_ticket = create_incident_ticket(incident_id)
    
    # Add both dev and ops members to incident
    assign_incident_team(jira_ticket, get_current_oncall())
    
    # Create war room in Slack
    create_incident_room(incident_id, jira_ticket)
    
    # Start incident timeline in GitLab
    create_incident_merge_request(incident_id, jira_ticket)

Knowledge Sharing Mechanisms

Post-Incident Learning:

# Post-incident process automation
post_incident_analysis:
  stage: learn
  when: manual
  script:
    - |
      # Generate incident timeline from GitLab and Jira
      python scripts/generate_incident_timeline.py \
        --incident-id $INCIDENT_ID \
        --jira-ticket $JIRA_TICKET
      
      # Schedule blameless postmortem
      python scripts/schedule_postmortem.py \
        --incident-id $INCIDENT_ID \
        --attendees "dev-team,ops-team,product-team"
      
      # Update knowledge base
      python scripts/update_runbooks.py \
        --incident-id $INCIDENT_ID \
        --lessons-learned "docs/postmortem-$INCIDENT_ID.md"

CALMS Implementation: A Year Later

Quantifiable Results

After one year of CALMS implementation, our transformation results were remarkable:

Technical Metrics:

  • Deployment frequency: From 1 per week to 15+ per day

  • Lead time: From 14 days to 4.1 days

  • Change failure rate: From 25% to 2.1%

  • MTTR: From 8 hours to 47 minutes

Cultural Metrics:

  • Cross-team collaboration score: 9.1/10 (up from 3.2/10)

  • Employee satisfaction: 8.5/10 (up from 5.1/10)

  • Knowledge sharing index: 85% (up from 25%)

  • Blame culture elimination: 100% (measured through incident retrospectives)

Business Impact:

  • Time to market: 70% reduction

  • Customer satisfaction: 40% improvement

  • System reliability: 99.8% uptime

  • Development velocity: 200% increase

The CALMS Maturity Journey

Here's a sequence diagram showing our maturity progression:

Lessons Learned: What I Wish I Knew Earlier

Start with Culture, Not Tools

My biggest lesson was that technology adoption without cultural change is futile. We tried implementing GitLab CI/CD before addressing team dynamics—it failed miserably. Only after establishing collaborative practices did our automation efforts succeed.

Measure What Matters to the Business

Initially, we tracked purely technical metrics. Adding business impact measurements (customer satisfaction, time-to-market, revenue impact) helped us gain stakeholder buy-in and demonstrate real value.

Make Sharing Non-Optional

Knowledge sharing didn't happen naturally. We had to make it part of our definition of done:

  • Documentation updates required for every feature

  • Cross-team code reviews mandatory

  • Post-incident learnings shared organization-wide

CALMS is Iterative, Not Linear

We initially tried to implement all components simultaneously. The sequential approach (Culture → Automation → Lean → Measurement → Sharing) worked much better for our team's maturity level.

Conclusion: CALMS as a DevOps Compass

Three years later, CALMS continues to guide our DevOps evolution. It's not just a framework—it's become our cultural DNA. The combination of Jira for collaborative planning and GitLab for integrated delivery has created a seamless experience where DevOps practices feel natural rather than forced.

The most significant change isn't in our tools or processes—it's in how we think about our work. We've moved from "my job" to "our mission," from blame to blameless improvement, and from heroics to sustainable practices.

If you're starting a DevOps transformation, I encourage you to use CALMS as your assessment framework. It will help you identify where you are, where you need to go, and most importantly, how to get there with your team intact and energized.

Remember: DevOps isn't about the tools you choose—it's about the culture you build. CALMS helps you build that culture systematically and sustainably.

What's your experience with CALMS or other DevOps frameworks? I'd love to hear how you've approached cultural transformation in your organization. Share your stories in the comments below.

PreviousUnderstanding Hybrid Identity - Federation in Microsoft EcosystemNextSAST vs DAST: Understanding the Differences

Last updated