Data Structures - Arrays, Slices, Maps

The Memory Leak That Cost Us $2,000

Month two with Go in production. Our log aggregation service was consuming 12GB of RAM and climbing. Every hour, memory usage increased by 500MB. After three days, the container OOMKilled and restarted.

I found the culprit:

var allLogs []string  // Growing forever!

func processLogBatch(logs []string) {
    allLogs = append(allLogs, logs...)  // Memory leak!
    // Process logs
    sendToElasticsearch(allLogs)
}

I was appending every batch to a global slice that never cleared. After processing millions of log lines, we were holding everything in memory.

The fix was understanding slices:

func processLogBatch(logs []string) {
    // Process only current batch
    sendToElasticsearch(logs)
    // logs goes out of scope, garbage collected
}

Memory usage dropped to 200MB stable. This article covers the slice internals I wish I'd understood on day one.

---

Arrays: Fixed-Size Collections

Arrays in Go have fixed size determined at compile time.

Array Declaration

var numbers [5]int  // Array of 5 integers
fmt.Println(numbers)  // [0 0 0 0 0] (zero values)

// Array literal
primes := [5]int{2, 3, 5, 7, 11}
fmt.Println(primes)  // [2 3 5 7 11]

// Let compiler count
primes := [...]int{2, 3, 5, 7, 11}  // Size is 5

Arrays Are Values

a := [3]int{1, 2, 3}
b := a  // Copy entire array!

b[0] = 99
fmt.Println(a)  // [1 2 3]  (unchanged)
fmt.Println(b)  // [99 2 3]

Key point: Arrays are copied by value, not by reference.

Array Limitations

var arr1 [5]int
var arr2 [10]int

// arr1 = arr2  // Error! Different types

// Arrays of different sizes are different types
func process(numbers [5]int) {
    // Can only pass [5]int arrays
}

Reality: I almost never use arrays directly. Slices are better for 99% of cases.

---

Slices: Dynamic Arrays

Slices are Go's most-used data structure. They're references to underlying arrays with dynamic size.

Slice Declaration

var numbers []int  // Slice (not array!)
fmt.Println(numbers)        // []
fmt.Println(numbers == nil) // true

// Slice literal
primes := []int{2, 3, 5, 7, 11}

// Using make
numbers := make([]int, 5)     // Length 5, capacity 5
fmt.Println(numbers)          // [0 0 0 0 0]

buffer := make([]byte, 0, 100)  // Length 0, capacity 100

Slice Internals

A slice has three components:

type slice struct {
    ptr *array  // Pointer to underlying array
    len int     // Current length
    cap int     // Capacity
}

Visual representation:

slice := []int{2, 3, 5, 7, 11}

slice: [ptr] [len=5] [cap=5]
        ↓
array:  [2][3][5][7][11]

Length vs Capacity

numbers := make([]int, 3, 5)  // len=3, cap=5

fmt.Println(len(numbers))  // 3
fmt.Println(cap(numbers))  // 5
fmt.Println(numbers)       // [0 0 0]

// Can access indices 0-2
numbers[0] = 1
numbers[1] = 2
numbers[2] = 3

// numbers[3] = 4  // Panic! Index out of range

---

Append: Growing Slices

Basic Append

var numbers []int  // nil slice

numbers = append(numbers, 1)
numbers = append(numbers, 2)
numbers = append(numbers, 3)

fmt.Println(numbers)  // [1 2 3]

Must assign back! append returns a new slice.

Append Multiple Values

numbers := []int{1, 2, 3}
numbers = append(numbers, 4, 5, 6)
fmt.Println(numbers)  // [1 2 3 4 5 6]

// Append another slice
more := []int{7, 8, 9}
numbers = append(numbers, more...)  // Spread operator
fmt.Println(numbers)  // [1 2 3 4 5 6 7 8 9]

Capacity Growth

This is where the memory leak started:

numbers := make([]int, 0)
fmt.Printf("len=%d cap=%d\n", len(numbers), cap(numbers))
// len=0 cap=0

for i := 0; i < 20; i++ {
    numbers = append(numbers, i)
    fmt.Printf("len=%d cap=%d\n", len(numbers), cap(numbers))
}

// Output:
// len=1 cap=1
// len=2 cap=2
// len=3 cap=4   <- Capacity doubled!
// len=4 cap=4
// len=5 cap=8   <- Doubled again
// len=8 cap=8
// len=9 cap=16  <- Keeps doubling

When capacity is exceeded, Go allocates a new array (usually double the size) and copies elements.

Pre-allocate When Possible

// Bad: Multiple allocations
var results []Result
for _, item := range items {
    results = append(results, process(item))
}

// Good: Single allocation
results := make([]Result, 0, len(items))
for _, item := range items {
    results = append(results, process(item))
}

This saved 40% memory allocations in my data processor.

---

Slicing Slices

Create Sub-slices

numbers := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}

slice1 := numbers[2:5]  // [2 3 4]
slice2 := numbers[:5]   // [0 1 2 3 4]
slice3 := numbers[5:]   // [5 6 7 8 9]
slice4 := numbers[:]    // [0 1 2 3 4 5 6 7 8 9] (copy reference)

Critical: Sub-slices share the underlying array!

numbers := []int{0, 1, 2, 3, 4, 5}
slice := numbers[2:5]  // [2 3 4]

slice[0] = 99

fmt.Println(numbers)  // [0 1 99 3 4 5]  <- Original changed!
fmt.Println(slice)    // [99 3 4]

The Leak I Had

// Read 1GB file
data, _ := os.ReadFile("huge.json")  // 1GB in memory

// Extract small part
header := data[:100]  // Only need 100 bytes

// Problem: 'header' still references entire 1GB array!
processHeader(header)

// Solution: Copy to new slice
header := make([]byte, 100)
copy(header, data[:100])  // Now 'data' can be garbage collected

---

Copy Function

source := []int{1, 2, 3, 4, 5}
dest := make([]int, len(source))

n := copy(dest, source)
fmt.Println(n)     // 5 (elements copied)
fmt.Println(dest)  // [1 2 3 4 5]

dest[0] = 99
fmt.Println(source)  // [1 2 3 4 5]  (unchanged)

Partial Copy

source := []int{1, 2, 3, 4, 5}
dest := make([]int, 3)

copy(dest, source)  // Copies min(len(dest), len(source))
fmt.Println(dest)   // [1 2 3]

Copy Overlapping Slices

numbers := []int{1, 2, 3, 4, 5}
copy(numbers[1:], numbers[:4])  // Shift left
fmt.Println(numbers)  // [1 1 2 3 4]

---

Maps: Key-Value Pairs

Map Declaration

// Nil map (can't assign to it!)
var ages map[string]int
// ages["Alice"] = 30  // Panic!

// Using make
ages := make(map[string]int)
ages["Alice"] = 30
ages["Bob"] = 25

// Map literal
ages := map[string]int{
    "Alice": 30,
    "Bob":   25,
    "Carol": 28,
}

Map Operations

// Add/Update
ages["Dave"] = 35

// Read
age := ages["Alice"]  // 30

// Read with existence check
age, exists := ages["Unknown"]
if exists {
    fmt.Println("Age:", age)
} else {
    fmt.Println("Not found")
}

// Delete
delete(ages, "Bob")

// Length
fmt.Println(len(ages))  // 3

Zero Value for Missing Keys

ages := map[string]int{
    "Alice": 30,
}

age := ages["Unknown"]  // 0 (zero value for int)
fmt.Println(age)        // 0

// Always check if needed
if age, exists := ages["Unknown"]; exists {
    fmt.Println("Age:", age)
} else {
    fmt.Println("Person not found")
}

Iterate Over Map

ages := map[string]int{
    "Alice": 30,
    "Bob":   25,
    "Carol": 28,
}

// Unordered iteration!
for name, age := range ages {
    fmt.Printf("%s is %d\n", name, age)
}

// Just keys
for name := range ages {
    fmt.Println(name)
}

Important: Map iteration order is randomized. Don't rely on order!

---

Real-World Example: Word Counter

func wordCount(text string) map[string]int {
    counts := make(map[string]int)
    
    words := strings.Fields(text)  // Split on whitespace
    
    for _, word := range words {
        word = strings.ToLower(word)
        counts[word]++  // Increment (zero value is 0)
    }
    
    return counts
}

text := "Go is awesome Go is fast Go is simple"
counts := wordCount(text)

for word, count := range counts {
    fmt.Printf("%s: %d\n", word, count)
}
// is: 3
// go: 3
// awesome: 1
// fast: 1
// simple: 1

---

Map of Slices

Common pattern for grouping:

type User struct {
    Name string
    Role string
}

users := []User{
    {"Alice", "admin"},
    {"Bob", "user"},
    {"Carol", "admin"},
    {"Dave", "user"},
}

// Group by role
byRole := make(map[string][]User)

for _, user := range users {
    byRole[user.Role] = append(byRole[user.Role], user)
}

fmt.Println(byRole["admin"])  // [{Alice admin} {Carol admin}]
fmt.Println(byRole["user"])   // [{Bob user} {Dave user}]

---

Maps Are Reference Types

original := map[string]int{"a": 1, "b": 2}
copy := original  // Both reference same map!

copy["c"] = 3

fmt.Println(original)  // map[a:1 b:2 c:3]  <- Changed!

To copy a map:

original := map[string]int{"a": 1, "b": 2}
copy := make(map[string]int)

for k, v := range original {
    copy[k] = v
}

copy["c"] = 3
fmt.Println(original)  // map[a:1 b:2]  (unchanged)

---

Practical Example: Cache Implementation

From my API service:

type Cache struct {
    data map[string]interface{}
    mu   sync.RWMutex
}

func NewCache() *Cache {
    return &Cache{
        data: make(map[string]interface{}),
    }
}

func (c *Cache) Get(key string) (interface{}, bool) {
    c.mu.RLock()
    defer c.mu.RUnlock()
    
    value, exists := c.data[key]
    return value, exists
}

func (c *Cache) Set(key string, value interface{}) {
    c.mu.Lock()
    defer c.mu.Unlock()
    
    c.data[key] = value
}

func (c *Cache) Delete(key string) {
    c.mu.Lock()
    defer c.mu.Unlock()
    
    delete(c.data, key)
}

// Usage
cache := NewCache()
cache.Set("user:123", User{Name: "Alice"})

if value, exists := cache.Get("user:123"); exists {
    user := value.(User)
    fmt.Println(user.Name)
}

---

Comparing Slices and Maps

You cannot use == to compare slices or maps:

a := []int{1, 2, 3}
b := []int{1, 2, 3}

// if a == b { }  // Error!

// Must compare manually
func equalSlices(a, b []int) bool {
    if len(a) != len(b) {
        return false
    }
    for i := range a {
        if a[i] != b[i] {
            return false
        }
    }
    return true
}

For maps:

func equalMaps(a, b map[string]int) bool {
    if len(a) != len(b) {
        return false
    }
    for k, v := range a {
        if b[k] != v {
            return false
        }
    }
    return true
}

Or use reflect.DeepEqual (slower):

import "reflect"

if reflect.DeepEqual(a, b) {
    // Equal
}

---

Common Pitfalls

1. Forgetting to Assign Append

numbers := []int{1, 2, 3}
append(numbers, 4)  // BUG! Result discarded

// Correct
numbers = append(numbers, 4)

2. Range Loop Variable Reuse

items := []int{1, 2, 3}
var results []*int

for _, item := range items {
    results = append(results, &item)  // BUG! Same address
}

// All point to same variable (last value)
for _, r := range results {
    fmt.Println(*r)  // 3 3 3
}

// Correct: Create new variable
for _, item := range items {
    item := item  // Shadow
    results = append(results, &item)
}

3. Modifying Map While Iterating

ages := map[string]int{"Alice": 30, "Bob": 25}

// Dangerous!
for name := range ages {
    if name == "Alice" {
        delete(ages, name)  // Modifying during iteration
    }
}

// Safer: Collect keys first
toDelete := []string{}
for name := range ages {
    if name == "Alice" {
        toDelete = append(toDelete, name)
    }
}
for _, name := range toDelete {
    delete(ages, name)
}

---

Your Challenge

Implement a simple in-memory database:

type Database struct {
    // Your implementation
}

// Implement these methods:
// - Set(key, value string)
// - Get(key string) (string, bool)
// - Delete(key string)
// - Keys() []string
// - Count() int
// - Clear()

// Bonus: Add expiration time for keys

---

Key Takeaways

1. Arrays are fixed size, slices are dynamic

2. Slices are references to underlying arrays

3. Append must be assigned back: s = append(s, v)

4. Pre-allocate slices when size is known for performance

5. Sub-slices share memory - can cause leaks

6. Maps are unordered and reference types

7. Check map existence with two-value assignment

8. Zero values: nil for slices/maps, 0 for missing map keys

---

What I Learned

The $2,000 memory leak taught me:

• Slices grow automatically but never shrink

• Understand capacity to avoid unnecessary allocations

• Copy when needed to break references

• Maps are perfect for fast lookups but use memory

Coming from Python's lists and dicts, Go's slices and maps felt similar. But the memory model is different - understanding references vs values prevented production bugs.

---

Next: Structs and Interfaces

In the next article, we'll explore Go's approach to types without classes: structs for data, interfaces for behavior, and composition over inheritance. You'll see how this simplifies design.