Tech With Htunn
  • Blog Content
  • 🤖Artificial Intelligence
    • 🧠Building an Intelligent Agent with Local LLMs and Azure OpenAI
    • 📊Revolutionizing IoT Monitoring: My Personal Journey with LLM-Powered Observability
  • 📘Core Concepts
    • 🔄Understanding DevSecOps
    • ⬅️Shifting Left in DevSecOps
    • 📦Understanding Containerization
    • ⚙️What is Site Reliability Engineering?
    • ⏱️Understanding Toil in SRE
    • 🔐What is Identity and Access Management?
    • 📊Microsoft Graph API: An Overview
    • 🔄Understanding Identity Brokers
  • 🔎Security Testing
    • 🔍SAST vs DAST: Understanding the Differences
    • 🧩Software Composition Analysis (SCA)
    • 📋Software Bill of Materials (SBOM)
    • 🧪Dependency Scanning in DevSecOps
    • 🐳Container Scanning in DevSecOps
  • 🔄CI/CD Pipeline
    • 🔁My Journey with Continuous Integration in DevOps
    • 🚀My Journey with Continuous Delivery and Deployment in DevOps
  • 🧮Fundamentals
    • 💾What is Data Engineering?
    • 🔄Understanding DataOps
    • 👷The Role of a Cloud Architect
    • 🏛️Cloud Native Architecture
    • 💻Cloud Native Applications
  • 🏛️Architecture & Patterns
    • 🏅Medallion Architecture in Data Engineering
    • 🔄ETL vs ELT Pipeline: Understanding the Differences
  • 🔒Authentication & Authorization
    • 🔑OAuth 2.0 vs OIDC: Key Differences
    • 🔐Understanding PKCE in OAuth 2.0
    • 🔄Service Provider vs Identity Provider Initiated SAML Flows
  • 📋Provisioning Standards
    • 📊SCIM in Identity and Access Management
    • 📡Understanding SCIM Streaming
  • 🏗️Design Patterns
    • ⚡Event-Driven Architecture
    • 🔒Web Application Firewalls
  • 📊Reliability Metrics
    • 💰Error Budgets in SRE
    • 📏SLA vs SLO vs SLI: Understanding the Differences
    • ⏱️Mean Time to Recovery (MTTR)
Powered by GitBook
On this page
  • What is Continuous Integration, Really?
  • The Day CI Changed My Development Life
  • Setting Up My First CI Pipeline with GitLab
  • Step 1: Creating the Pipeline Configuration
  • Step 2: Setting Up the Preparation Stage
  • Step 3: Building the Application
  • Step 4: Implementing Automated Testing
  • Step 5: Adding Code Quality Checks
  • Real-World Results from My CI Implementation
  • My Advice for Getting Started with CI
  • The Unexpected Benefits of CI
  • Final Thoughts
  1. CI/CD Pipeline

My Journey with Continuous Integration in DevOps

When I started my career as a developer, our integration process was what I'd now call "discontinuous integration." We'd code in isolation for weeks, then spend days—sometimes weeks—dealing with merge conflicts and integration bugs. Today, I want to share how implementing Continuous Integration transformed my team's development process and how you can achieve the same with GitLab.

What is Continuous Integration, Really?

After years working in software development, I've come to understand that Continuous Integration isn't just a technical practice—it's a fundamental shift in how we work together on code.

In my experience, Continuous Integration (CI) means breaking down the walls between developers by integrating our code changes into a shared repository multiple times daily. Each integration is verified automatically through building the application and running tests. This approach has completely changed how my teams collaborate.

The Day CI Changed My Development Life

I still remember the turning point. Our team was working on a critical e-commerce platform, and we kept stepping on each other's toes. One developer's changes would constantly break another's code, and we'd spend entire sprints just fixing integration issues.

Then our CTO mandated that everyone integrate code at least twice daily. I was skeptical—wouldn't this create even more conflicts? But we set up automated testing with GitLab CI, and something magical happened: our integration problems practically disappeared overnight. Here's why:

  1. We started making smaller, focused changes — When you know you'll commit soon, you don't build massive, sweeping changes

  2. We got immediate feedback — No more waiting days to discover a bug you introduced

  3. We caught conflicts early — Merging five lines of code is infinitely easier than merging 500

  4. Our code quality improved — With automated tests catching issues immediately, quality became built-in

Setting Up My First CI Pipeline with GitLab

After trying several CI tools, I settled on GitLab for its simplicity and power. Here's exactly how I set up a CI pipeline for our Node.js application—a process I've now replicated across dozens of projects:

Step 1: Creating the Pipeline Configuration

First, I added a .gitlab-ci.yml file to the root of our repository. This defines the entire CI process:

# This is the backbone of our CI process
stages:
  - prepare
  - build
  - test
  - quality

variables:
  NODE_ENV: 'test'

# Cache node_modules to speed up builds
cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - node_modules/

A hard-learned tip: I use ${CI_COMMIT_REF_SLUG} as the cache key to ensure each branch gets its own cache, avoiding strange dependency conflicts between branches.

Step 2: Setting Up the Preparation Stage

install_dependencies:
  stage: prepare
  script:
    - echo "Installing dependencies..."
    - npm ci
  artifacts:
    paths:
      - node_modules/

Lesson learned: I switched from npm install to npm ci after numerous issues with inconsistent dependency versions. The ci command guarantees reproducible builds by strictly following package-lock.json.

Step 3: Building the Application

build_app:
  stage: build
  script:
    - echo "Building application..."
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 day

I always set artifacts to expire to avoid cluttering GitLab's storage. One day is usually enough for CI purposes.

Step 4: Implementing Automated Testing

unit_tests:
  stage: test
  script:
    - echo "Running unit tests..."
    - npm run test:unit
  artifacts:
    reports:
      junit: junit-test-results.xml
    
integration_tests:
  stage: test
  script:
    - echo "Running integration tests..."
    - npm run test:integration
    
coverage:
  stage: test
  script:
    - echo "Checking test coverage..."
    - npm run test:coverage
  coverage: /All\sfiles.*?\s+(\d+.\d+)/
  artifacts:
    paths:
      - coverage/

I split tests into separate jobs so a single slow test type doesn't hold up the entire pipeline, and failing unit tests can provide fast feedback without waiting for integration tests.

Step 5: Adding Code Quality Checks

lint:
  stage: quality
  script:
    - echo "Linting code..."
    - npm run lint
    
code_quality:
  stage: quality
  script:
    - echo "Analyzing code quality..."
    - npm run analyze
  allow_failure: true  # Don't block the pipeline

I initially set allow_failure: true on quality checks when introducing them to an existing codebase. This lets the team see issues without blocking deployments while they clean things up.

Real-World Results from My CI Implementation

The impact of proper CI on my teams has been tremendous:

  1. Integration time dropped by 87%: We went from spending days integrating changes to minutes.

  2. Bugs detected earlier: On average, we catch bugs within 45 minutes of introduction versus days or weeks later.

  3. Knowledge sharing increased: Frequent integrations meant developers saw each other's code more often, improving collective code ownership.

  4. Team stress reduced: The constant fear of "merge day" disappeared completely.

My Advice for Getting Started with CI

Based on my experience implementing CI across multiple teams, here's what I recommend:

  1. Start simple: Begin with just building and running basic tests—you can add complexity later.

  2. Make it fast: If your CI pipeline takes too long, developers will avoid using it. Optimize for speed.

  3. Fix broken builds immediately: Establish a team rule that fixing the build takes priority over new features.

  4. Celebrate success: When the CI catches a bug that would have made it to production, highlight it!

  5. Iterate and improve: Add more automated tests and quality checks as your team grows more comfortable with CI.

The Unexpected Benefits of CI

Beyond the technical improvements, CI brought cultural benefits I hadn't anticipated:

  • Increased trust: Team members trusted each other more when integration issues decreased

  • More experimentation: Developers became more willing to try new approaches, knowing tests would catch issues

  • Shared ownership: The codebase truly became "ours" not "mine and yours"

  • Onboarding improved: New team members could contribute confidently on day one, protected by our test suite

Final Thoughts

Implementing Continuous Integration was one of the most transformative changes in my development career. It shifted us from a group of individual developers to a truly collaborative team building software together.

If you're still integrating code infrequently, I encourage you to try CI. Start small with GitLab CI/CD, integrate often, and watch how it transforms not just your code, but your entire development culture.

Remember: CI isn't just about catching bugs earlier—it's about creating the foundation for a truly collaborative development process where the whole is greater than the sum of its parts.

PreviousContainer Scanning in DevSecOpsNextMy Journey with Continuous Delivery and Deployment in DevOps

Last updated 2 days ago

🔄
🔁