# Microkernel Architecture

## How VS Code Taught Me to Think About Plugins

I have used VS Code every day for years. What struck me about it architecturally — when I started paying attention to how it worked — is that VS Code itself knows almost nothing about any specific programming language, debugger, or tool. All of that lives in extensions. The editor is a minimal core that exposes extension points; the ecosystem is the functionality.

That is microkernel architecture in practice: a small, stable core (the kernel) that handles only the most essential responsibilities, and a system of plugins (extensions) that add capabilities without modifying the core.

I have applied this pattern myself when building a data pipeline tool where the processing steps were extensible — different teams needed to add their own transformations without modifying the shared platform. The microkernel pattern gave them a contract to implement and a way to register their plugin.

## Table of Contents

* [What Is Microkernel Architecture?](#what-is-microkernel-architecture)
* [Core vs Plugins](#core-vs-plugins)
* [Plugin Registration and Discovery](#plugin-registration-and-discovery)
* [Practical Example: Extensible Data Pipeline in TypeScript](#practical-example-extensible-data-pipeline-in-typescript)
* [Real-World Examples](#real-world-examples)
* [When to Use Microkernel](#when-to-use-microkernel)
* [Challenges](#challenges)
* [Lessons Learned](#lessons-learned)

***

## What Is Microkernel Architecture?

The microkernel pattern consists of two primary components:

* **Core System (the kernel):** Minimal functionality required for the system to run. Handles plugin lifecycle, communication, and shared utilities. Stable and rarely changed.
* **Plug-in Modules:** Independent components that extend the core's functionality. Each plugin can be added, removed, or updated without changing the core or other plugins.

{% @mermaid/diagram content="graph TB
subgraph Core\["Core System (Kernel)"]
REGISTRY\[Plugin Registry]
LIFECYCLE\[Plugin Lifecycle Manager]
EVENTBUS\[Internal Event Bus]
SHARED\[Shared Services<br/>Logging, Config, Auth]
end

```
subgraph Plugins["Plug-in Modules"]
    P1[Plugin A<br/>CSV Transformer]
    P2[Plugin B<br/>JSON Transformer]
    P3[Plugin C<br/>Database Writer]
    P4[Plugin D<br/>S3 Writer]
end

REGISTRY --> P1
REGISTRY --> P2
REGISTRY --> P3
REGISTRY --> P4

P1 --> EVENTBUS
P2 --> EVENTBUS
P3 --> EVENTBUS
P4 --> EVENTBUS

style Core fill:#ffeaa7" %}
```

Plugins communicate with the core through a defined interface. Plugins typically do not communicate with each other directly — they go through the core's event bus or registry.

***

## Core vs Plugins

| Responsibility                       | Lives In | Example                            |
| ------------------------------------ | -------- | ---------------------------------- |
| Plugin registration and discovery    | Core     | `PluginRegistry` class             |
| Plugin lifecycle (load, start, stop) | Core     | `PluginManager`                    |
| Shared utilities all plugins use     | Core     | Logger, config, auth               |
| Application-specific processing      | Plugin   | `CSVTransformer`, `JSONNormaliser` |
| Domain-specific output targets       | Plugin   | `PostgresWriter`, `S3Uploader`     |
| Third-party integrations             | Plugin   | `SlackNotifier`, `WebhookEmitter`  |

The core should be small enough that you can understand it completely. If the core keeps growing to accommodate new requirements, either the boundary is drawn wrong or the new requirements belong in a plugin.

***

## Plugin Registration and Discovery

Plugins register themselves against a defined interface. The core discovers and manages them through a registry.

Two common patterns:

### Static Registration

Plugins are registered at startup in a configuration file or code:

```typescript
// plugin-registry.ts
const plugins: Plugin[] = [
  new CSVTransformerPlugin(),
  new JSONNormaliserPlugin(),
  new PostgresWriterPlugin(config.DB_URL),
];
core.registerAll(plugins);
```

### Dynamic Discovery

Plugins are discovered at runtime by scanning a directory or reading a manifest:

```typescript
// auto-discovery: loads any file in ./plugins/ that exports a Plugin class
import { readdirSync } from 'fs';
import { join } from 'path';

async function discoverPlugins(pluginsDir: string): Promise<Plugin[]> {
  const plugins: Plugin[] = [];
  const entries = readdirSync(pluginsDir).filter(f => f.endsWith('.js'));
  for (const entry of entries) {
    const mod = await import(join(pluginsDir, entry));
    if (mod.default && typeof mod.default === 'function') {
      plugins.push(new mod.default());
    }
  }
  return plugins;
}
```

***

## Practical Example: Extensible Data Pipeline in TypeScript

Here is a simplified version of the data pipeline tool I built. The core processes records; plugins handle the transformations and outputs.

```typescript
// core/plugin.interface.ts — the contract all plugins must implement

export interface PipelinePlugin {
  readonly name: string;
  readonly type: 'transformer' | 'writer';

  /**
   * Called once when the plugin is registered.
   */
  initialize(config: Record<string, unknown>): Promise<void>;

  /**
   * Transformers: receives a record, returns transformed record.
   * Writers: receives a record, persists it somewhere.
   */
  process(record: Record<string, unknown>): Promise<Record<string, unknown>>;

  /**
   * Called on shutdown — close connections, flush buffers.
   */
  teardown(): Promise<void>;
}
```

```typescript
// core/pipeline-core.ts — the kernel

import { PipelinePlugin } from './plugin.interface';

export class PipelineCore {
  private readonly transformers: PipelinePlugin[] = [];
  private readonly writers: PipelinePlugin[] = [];

  register(plugin: PipelinePlugin): void {
    if (plugin.type === 'transformer') {
      this.transformers.push(plugin);
    } else {
      this.writers.push(plugin);
    }
    console.log(`Registered plugin: ${plugin.name} (${plugin.type})`);
  }

  async process(records: Record<string, unknown>[]): Promise<void> {
    for (let record of records) {
      // Apply transformers in order
      for (const transformer of this.transformers) {
        record = await transformer.process(record);
      }
      // Write to all writers in parallel
      await Promise.all(this.writers.map(w => w.process(record)));
    }
  }

  async shutdown(): Promise<void> {
    const allPlugins = [...this.transformers, ...this.writers];
    await Promise.all(allPlugins.map(p => p.teardown()));
  }
}
```

```typescript
// plugins/transformers/normalize-timestamps.ts — a transformer plugin

import { PipelinePlugin } from '../../core/plugin.interface';

export class NormalizeTimestampsPlugin implements PipelinePlugin {
  readonly name = 'normalize-timestamps';
  readonly type = 'transformer' as const;
  private timezone = 'UTC';

  async initialize(config: Record<string, unknown>): Promise<void> {
    this.timezone = (config.timezone as string) ?? 'UTC';
  }

  async process(record: Record<string, unknown>): Promise<Record<string, unknown>> {
    const result = { ...record };
    for (const [key, value] of Object.entries(result)) {
      if (typeof value === 'string' && /^\d{4}-\d{2}-\d{2}/.test(value)) {
        result[key] = new Date(value).toISOString();
      }
    }
    return result;
  }

  async teardown(): Promise<void> {
    // Nothing to clean up
  }
}
```

```typescript
// plugins/writers/postgres-writer.ts — a writer plugin

import { Pool } from 'pg';
import { PipelinePlugin } from '../../core/plugin.interface';

export class PostgresWriterPlugin implements PipelinePlugin {
  readonly name = 'postgres-writer';
  readonly type = 'writer' as const;
  private pool!: Pool;
  private tableName = 'pipeline_records';

  async initialize(config: Record<string, unknown>): Promise<void> {
    this.pool = new Pool({ connectionString: config.dbUrl as string });
    this.tableName = (config.tableName as string) ?? 'pipeline_records';
  }

  async process(record: Record<string, unknown>): Promise<Record<string, unknown>> {
    const keys = Object.keys(record);
    const values = Object.values(record);
    const placeholders = keys.map((_, i) => `$${i + 1}`).join(', ');

    await this.pool.query(
      `INSERT INTO ${this.tableName} (${keys.join(', ')}) VALUES (${placeholders})`,
      values
    );
    return record;
  }

  async teardown(): Promise<void> {
    await this.pool.end();
  }
}
```

```typescript
// main.ts — wiring the core with plugins

import { PipelineCore } from './core/pipeline-core';
import { NormalizeTimestampsPlugin } from './plugins/transformers/normalize-timestamps';
import { PostgresWriterPlugin } from './plugins/writers/postgres-writer';
import { S3WriterPlugin } from './plugins/writers/s3-writer';

async function main() {
  const core = new PipelineCore();

  const timestampPlugin = new NormalizeTimestampsPlugin();
  await timestampPlugin.initialize({ timezone: 'Asia/Bangkok' });

  const postgresPlugin = new PostgresWriterPlugin();
  await postgresPlugin.initialize({
    dbUrl: process.env.DATABASE_URL,
    tableName: 'sensor_events',
  });

  const s3Plugin = new S3WriterPlugin();
  await s3Plugin.initialize({
    bucket: process.env.S3_BUCKET,
    prefix: 'pipeline/',
  });

  core.register(timestampPlugin);   // transformer
  core.register(postgresPlugin);    // writer
  core.register(s3Plugin);          // writer

  const records = await fetchRecordsFromSource();
  await core.process(records);
  await core.shutdown();
}

main().catch(console.error);
```

To add a new output target (Elasticsearch, webhook, BigQuery), I write a new plugin implementing `PipelinePlugin`. I do not touch `PipelineCore` or any existing plugin.

***

## Real-World Examples

| System          | Core                          | Plugins                                         |
| --------------- | ----------------------------- | ----------------------------------------------- |
| **VS Code**     | Editor, layout, extension API | Language servers, debuggers, themes, formatters |
| **webpack**     | Module bundler core           | Loaders, plugins for transforms and outputs     |
| **Eclipse IDE** | Platform, OSGi runtime        | Language tools, debuggers, build integrations   |
| **Jenkins**     | Build orchestrator            | SCM integrations, build steps, notifications    |
| **ESLint**      | Linting engine                | Rules, plugins, formatters                      |

All of these follow the same principle: stable core + extensible plugin system. Adding functionality does not require modifying the core.

***

## When to Use Microkernel

* **The system needs to be extended by different teams or third parties** without modifying the central codebase
* **Features have very different lifecycles** — some parts change weekly, others are stable for years
* **The system is a platform**, not just an application — it needs to support use cases not known at build time
* **Plugin isolation is important** — one plugin breaking should not bring down the core
* **A CLI tool or development tool** that needs to be composable (think `prettier`, `eslint`, `babel`)

***

## Challenges

* **Plugin contract stability.** Once a plugin interface is public, changing it breaks all plugins. Design the interface carefully and version it explicitly.
* **Plugin discovery and loading in production.** Dynamic plugin loading can introduce security risks — ensure plugins come from trusted sources.
* **Inter-plugin dependencies.** When plugins need to communicate with each other, the architecture gets complex. Try to route all communication through the core.
* **Error isolation.** A misbehaving plugin should not crash the core. Consider sandboxing or error boundaries around plugin execution.

***

## Lessons Learned

* **Start with the minimum viable core.** I have made the mistake of over-engineering the core with features that "plugins will probably need." Ship the minimal core first; extend it only when a concrete plugin requires it.
* **The plugin interface is a public API.** Treat it with the same care as a public REST API — document it, version it, do not break it.
* **Plugins should be independently deployable.** If adding a plugin requires rebuilding the core, the boundary is not clean enough.
* **TypeScript interfaces are ideal for defining plugin contracts.** Strong typing ensures plugins implement the full contract and gives IDE support to plugin authors.