PowerShell Pipeline

The Moment I Understood Objects

I was helping a colleague who needed to find all files larger than 100MB on a file server to free up space. Coming from Linux, his first instinct was to reach for find, grep, and awk. I suggested PowerShell:

Get-ChildItem -Recurse | Where-Object {$_.Length -gt 100MB} | Sort-Object Length -Descending

He was skeptical. "But how do you know there's a Length property?" he asked. I showed him:

Get-ChildItem | Get-Member

His eyes widened as he saw the complete list of properties and methods available on file objects. No parsing strings with regular expressions. No splitting columns with awk. Just structured objects with accessible properties.

That's when he got it. PowerShell doesn't pass text between commands - it passes objects. And that changes everything.

What is the Pipeline?

The pipeline (|) is PowerShell's way of passing output from one cmdlet to another. But unlike Unix/Linux pipes that pass text streams, PowerShell pipes pass .NET objects.

Traditional Text-Based Pipes (Bash/Linux)

# Linux: text streams requiring parsing
ps aux | grep "python" | awk '{print $2, $11}'

You're working with text, so you need tools like grep, awk, sed to extract what you need.

PowerShell Object-Based Pipeline

# PowerShell: objects with properties
Get-Process | Where-Object {$_.Name -like "*python*"} | Select-Object Id, Name

You're working with process objects that have properties you can directly access. No parsing required.

How the Pipeline Works

When you pipe cmdlets together, the output of one becomes the input of the next:

Get-Process | Sort-Object CPU -Descending | Select-Object -First 10

Flow:

1. Get-Process outputs process objects

2. Sort-Object receives those objects, sorts by CPU property

3. Select-Object receives sorted objects, takes first 10

4. Final 10 objects displayed to screen

Visualizing Object Flow

# Step 1: Get processes (returns objects)
$step1 = Get-Process

# Step 2: Sort by CPU (returns sorted objects)
$step2 = $step1 | Sort-Object CPU -Descending

# Step 3: Take first 5 (returns 5 objects)
$step3 = $step2 | Select-Object -First 5

# All in one pipeline
Get-Process | Sort-Object CPU -Descending | Select-Object -First 5

Both approaches give the same result, but the pipeline is cleaner.

Pipeline Fundamentals

Basic Pipeline Syntax

Command1 | Command2 | Command3

Each command processes output from the previous command.

The Pipeline Variable: $_

Inside a pipeline, $_ represents the current object:

# $_ is the current process object
Get-Process | Where-Object {$_.CPU -gt 100}

# $_ is the current file object
Get-ChildItem | Where-Object {$_.Length -gt 1MB}

# $_ is the current service object
Get-Service | Where-Object {$_.Status -eq 'Running'}

In PowerShell 7+, you can also use $PSItem instead of $_ (they're identical).

Getting to Know Objects: Get-Member

Before working with objects in the pipeline, discover their properties and methods:

# See all properties and methods of processes
Get-Process | Get-Member

# See only properties
Get-Process | Get-Member -MemberType Property

# See only methods
Get-Process | Get-Member -MemberType Method

# See properties and methods of files
Get-ChildItem | Get-Member

# See properties of services
Get-Service | Get-Member

I use Get-Member constantly when exploring new cmdlets or modules.

Essential Pipeline Cmdlets

Where-Object: Filtering

Filter objects based on conditions:

# Processes using more than 100MB of memory
Get-Process | Where-Object {$_.WorkingSet -gt 100MB}

# Running services only
Get-Service | Where-Object {$_.Status -eq 'Running'}

# Files modified in the last 7 days
Get-ChildItem | Where-Object {$_.LastWriteTime -gt (Get-Date).AddDays(-7)}

# .txt files larger than 1KB
Get-ChildItem | Where-Object {$_.Extension -eq '.txt' -and $_.Length -gt 1KB}

# Multiple conditions with -or
Get-Service | Where-Object {$_.Status -eq 'Running' -or $_.StartType -eq 'Automatic'}

Select-Object: Choosing Properties

Select specific properties or limit results:

# Select specific properties
Get-Process | Select-Object Name, CPU, WorkingSet

# Select first 10 items
Get-Process | Select-Object -First 10

# Select last 5 items
Get-ChildItem | Select-Object -Last 5

# Select unique values
Get-Process | Select-Object -Unique ProcessName

# Exclude certain properties
Get-Service | Select-Object * -ExcludeProperty RequiredServices, DependentServices

Calculated Properties

Create custom properties on the fly:

# Convert bytes to MB
Get-Process | Select-Object Name, 
    @{Name='MemoryMB'; Expression={$_.WorkingSet / 1MB}}

# Format dates
Get-ChildItem | Select-Object Name,
    @{Name='Modified'; Expression={$_.LastWriteTime.ToString('yyyy-MM-dd')}}

# Multiple calculated properties
Get-Process | Select-Object Name,
    @{Name='MemoryMB'; Expression={[math]::Round($_.WorkingSet / 1MB, 2)}},
    @{Name='CPUSeconds'; Expression={[math]::Round($_.CPU, 2)}}

Sort-Object: Ordering

Sort objects by properties:

# Sort by single property
Get-Process | Sort-Object CPU

# Sort descending
Get-Process | Sort-Object CPU -Descending

# Sort by multiple properties
Get-ChildItem | Sort-Object Extension, Name

# Sort with mixed order
Get-Process | Sort-Object -Property @{Expression='WorkingSet'; Descending=$true}, Name

Measure-Object: Statistics

Calculate statistics on object properties:

# Count items
Get-ChildItem | Measure-Object

# Sum file sizes
Get-ChildItem | Measure-Object -Property Length -Sum

# Get average, min, max
Get-Process | Measure-Object -Property WorkingSet -Average -Minimum -Maximum

# Multiple statistics
Get-ChildItem *.txt | Measure-Object -Property Length -Sum -Average -Maximum

Group-Object: Grouping

Group objects by property values:

# Group processes by company
Get-Process | Group-Object Company

# Group files by extension
Get-ChildItem | Group-Object Extension

# Group services by status
Get-Service | Group-Object Status

# Group with count and custom formatting
Get-Process | Group-Object Company | Select-Object Count, Name

Common Pipeline Patterns

Get → Filter → Select

The most common pattern:

# Get all processes, filter high CPU, select specific properties
Get-Process | 
    Where-Object {$_.CPU -gt 50} | 
    Select-Object Name, CPU, WorkingSet

# Get files, filter by size, select name and size
Get-ChildItem | 
    Where-Object {$_.Length -gt 1MB} | 
    Select-Object Name, Length

Get → Filter → Sort → Select

Add sorting to the mix:

# Top 10 memory-consuming processes
Get-Process | 
    Where-Object {$_.WorkingSet -gt 10MB} |
    Sort-Object WorkingSet -Descending |
    Select-Object -First 10 Name, WorkingSet

# Recently modified large files
Get-ChildItem | 
    Where-Object {$_.Length -gt 100MB} |
    Sort-Object LastWriteTime -Descending |
    Select-Object Name, Length, LastWriteTime

Get → Group → Select

Grouping and counting:

# Count files by extension
Get-ChildItem -Recurse | 
    Group-Object Extension | 
    Select-Object Count, Name |
    Sort-Object Count -Descending

# Count processes by company
Get-Process | 
    Group-Object Company | 
    Select-Object Count, Name |
    Sort-Object Count -Descending

Get → Measure → Format

Statistics and display:

# Total size of files in directory
$totalSize = Get-ChildItem | 
    Measure-Object -Property Length -Sum

Write-Host "Total size: $([math]::Round($totalSize.Sum / 1GB, 2)) GB"

# Average memory usage
$avgMemory = Get-Process | 
    Measure-Object -Property WorkingSet -Average

Write-Host "Average memory: $([math]::Round($avgMemory.Average / 1MB, 2)) MB"

Real-World Pipeline Examples

Finding Large Files

When disk space was running low on a file server:

Get-ChildItem -Path "C:\Users" -Recurse -File -ErrorAction SilentlyContinue |
    Where-Object {$_.Length -gt 500MB} |
    Sort-Object Length -Descending |
    Select-Object FullName, 
        @{Name='SizeMB'; Expression={[math]::Round($_.Length / 1MB, 2)}},
        LastWriteTime |
    Export-Csv -Path "LargeFiles.csv" -NoTypeInformation

Found 2.5TB of old backup files nobody knew existed.

Analyzing Process Resource Usage

Troubleshooting performance issues:

Get-Process |
    Where-Object {$_.WorkingSet -gt 100MB} |
    Sort-Object WorkingSet -Descending |
    Select-Object -First 20 Name,
        @{Name='MemoryGB'; Expression={[math]::Round($_.WorkingSet / 1GB, 2)}},
        @{Name='CPUTime'; Expression={$_.CPU}} |
    Format-Table -AutoSize

Checking Service Status

Monitoring critical services across servers:

$criticalServices = 'W32Time', 'EventLog', 'WinRM', 'wuauserv'

Get-Service -Name $criticalServices |
    Where-Object {$_.Status -ne 'Running'} |
    Select-Object Name, Status, StartType |
    Format-Table -AutoSize

If any critical service isn't running, it shows up immediately.

File Organization Report

Understanding directory structure:

Get-ChildItem -Recurse -File |
    Group-Object Extension |
    Select-Object @{Name='Extension'; Expression={if($_.Name) {$_.Name} else {'No Extension'}}},
                  Count,
                  @{Name='TotalSizeMB'; Expression={
                      [math]::Round(($_.Group | Measure-Object -Property Length -Sum).Sum / 1MB, 2)
                  }} |
    Sort-Object TotalSizeMB -Descending |
    Format-Table -AutoSize

Shows file types and space consumption - invaluable for cleanup planning.

Pipeline Performance Considerations

ForEach-Object vs foreach Loop

Pipeline approach (slower but more convenient):

Get-ChildItem | ForEach-Object {
    $_.Name.ToUpper()
}

Loop approach (faster for large datasets):

$files = Get-ChildItem
foreach ($file in $files) {
    $file.Name.ToUpper()
}

For small datasets (< 1000 items), use pipeline for readability. For large datasets, use loops for performance.

Early Filtering

Filter as early as possible:

# Bad - retrieves everything then filters
Get-ChildItem -Recurse | Where-Object {$_.Extension -eq '.txt'}

# Good - filters during retrieval
Get-ChildItem -Recurse -Filter "*.txt"

# Bad - gets all processes then filters
Get-Process | Where-Object {$_.Name -eq 'pwsh'}

# Good - filters during retrieval
Get-Process -Name 'pwsh'

Avoid Format-* in Pipeline

Format cmdlets (Format-Table, Format-List) convert objects to formatting instructions. Don't use them mid-pipeline:

# Bad - Format-Table breaks the pipeline
Get-Process | Format-Table | Where-Object {$_.CPU -gt 50}  # Won't work

# Good - Format-Table at the end only
Get-Process | Where-Object {$_.CPU -gt 50} | Format-Table

Common Pitfalls

1. Using Format-* Too Early

Problem: Get-Process | Format-Table | Export-Csv fails Solution: Format cmdlets should always be last:

Get-Process | Export-Csv process.csv  # Export first
Get-Content process.csv | Format-Table  # Format when viewing

2. Forgetting to Use Get-Member

Problem: Guessing property names incorrectly Solution: Always check with Get-Member first:

Get-Process | Get-Member  # See available properties

3. Inefficient Filtering

Problem: Filtering after retrieving all data Solution: Filter during data retrieval when possible

4. Breaking the Pipeline

Problem: Trying to use pipeline output incorrectly Solution: Understand what each cmdlet outputs:

# Check what's being passed
Get-Process | ForEach-Object { $_ | Get-Member }

Key Takeaways

• PowerShell pipes objects, not text - this is fundamental

• $_ represents the current object in the pipeline

• Get-Member reveals properties and methods available

• Common pattern: Get → Filter → Sort → Select

• Filter early for better performance

• Format cmdlets should always be last in the pipeline

• Calculated properties create custom fields on the fly

• Pipeline is readable - each step is clear and purposeful

What You've Learned

✅ How PowerShell's object-based pipeline differs from text pipes ✅ Using $_ to reference current pipeline objects ✅ Essential pipeline cmdlets: Where, Select, Sort, Measure, Group ✅ Common pipeline patterns for filtering and transforming data ✅ Real-world pipeline examples for practical tasks ✅ Performance considerations for large datasets ✅ Avoiding common pipeline pitfalls

Next Steps

Now that you master the pipeline, you're ready to work with PowerShell objects in depth. In Working with PowerShell Objects, you'll learn:

• Deep dive into object properties and methods

• Advanced filtering techniques

• Object manipulation and transformation

• Working with complex nested objects

• Custom object creation

Understanding objects is key to PowerShell mastery - let's go deeper.

---

Ready to master objects? Continue to Working with PowerShell Objects →