Microkernel Architecture

How VS Code Taught Me to Think About Plugins

I have used VS Code every day for years. What struck me about it architecturally — when I started paying attention to how it worked — is that VS Code itself knows almost nothing about any specific programming language, debugger, or tool. All of that lives in extensions. The editor is a minimal core that exposes extension points; the ecosystem is the functionality.

That is microkernel architecture in practice: a small, stable core (the kernel) that handles only the most essential responsibilities, and a system of plugins (extensions) that add capabilities without modifying the core.

I have applied this pattern myself when building a data pipeline tool where the processing steps were extensible — different teams needed to add their own transformations without modifying the shared platform. The microkernel pattern gave them a contract to implement and a way to register their plugin.

Table of Contents

• What Is Microkernel Architecture?

• Core vs Plugins

• Plugin Registration and Discovery

• Practical Example: Extensible Data Pipeline in TypeScript

• Real-World Examples

• When to Use Microkernel

• Challenges

• Lessons Learned

---

What Is Microkernel Architecture?

The microkernel pattern consists of two primary components:

• Core System (the kernel): Minimal functionality required for the system to run. Handles plugin lifecycle, communication, and shared utilities. Stable and rarely changed.

• Plug-in Modules: Independent components that extend the core's functionality. Each plugin can be added, removed, or updated without changing the core or other plugins.

Plugins communicate with the core through a defined interface. Plugins typically do not communicate with each other directly — they go through the core's event bus or registry.

---

Core vs Plugins

Responsibility	Lives In	Example
Plugin registration and discovery	Core	PluginRegistry class
Plugin lifecycle (load, start, stop)	Core	PluginManager
Shared utilities all plugins use	Core	Logger, config, auth
Application-specific processing	Plugin	CSVTransformer, JSONNormaliser
Domain-specific output targets	Plugin	PostgresWriter, S3Uploader
Third-party integrations	Plugin	SlackNotifier, WebhookEmitter

The core should be small enough that you can understand it completely. If the core keeps growing to accommodate new requirements, either the boundary is drawn wrong or the new requirements belong in a plugin.

---

Plugin Registration and Discovery

Plugins register themselves against a defined interface. The core discovers and manages them through a registry.

Two common patterns:

Static Registration

Plugins are registered at startup in a configuration file or code:

// plugin-registry.ts
const plugins: Plugin[] = [
  new CSVTransformerPlugin(),
  new JSONNormaliserPlugin(),
  new PostgresWriterPlugin(config.DB_URL),
];
core.registerAll(plugins);

Dynamic Discovery

Plugins are discovered at runtime by scanning a directory or reading a manifest:

// auto-discovery: loads any file in ./plugins/ that exports a Plugin class
import { readdirSync } from 'fs';
import { join } from 'path';

async function discoverPlugins(pluginsDir: string): Promise<Plugin[]> {
  const plugins: Plugin[] = [];
  const entries = readdirSync(pluginsDir).filter(f => f.endsWith('.js'));
  for (const entry of entries) {
    const mod = await import(join(pluginsDir, entry));
    if (mod.default && typeof mod.default === 'function') {
      plugins.push(new mod.default());
    }
  }
  return plugins;
}

---

Practical Example: Extensible Data Pipeline in TypeScript

Here is a simplified version of the data pipeline tool I built. The core processes records; plugins handle the transformations and outputs.

// core/plugin.interface.ts — the contract all plugins must implement

export interface PipelinePlugin {
  readonly name: string;
  readonly type: 'transformer' | 'writer';

  /**
   * Called once when the plugin is registered.
   */
  initialize(config: Record<string, unknown>): Promise<void>;

  /**
   * Transformers: receives a record, returns transformed record.
   * Writers: receives a record, persists it somewhere.
   */
  process(record: Record<string, unknown>): Promise<Record<string, unknown>>;

  /**
   * Called on shutdown — close connections, flush buffers.
   */
  teardown(): Promise<void>;
}

// core/pipeline-core.ts — the kernel

import { PipelinePlugin } from './plugin.interface';

export class PipelineCore {
  private readonly transformers: PipelinePlugin[] = [];
  private readonly writers: PipelinePlugin[] = [];

  register(plugin: PipelinePlugin): void {
    if (plugin.type === 'transformer') {
      this.transformers.push(plugin);
    } else {
      this.writers.push(plugin);
    }
    console.log(`Registered plugin: ${plugin.name} (${plugin.type})`);
  }

  async process(records: Record<string, unknown>[]): Promise<void> {
    for (let record of records) {
      // Apply transformers in order
      for (const transformer of this.transformers) {
        record = await transformer.process(record);
      }
      // Write to all writers in parallel
      await Promise.all(this.writers.map(w => w.process(record)));
    }
  }

  async shutdown(): Promise<void> {
    const allPlugins = [...this.transformers, ...this.writers];
    await Promise.all(allPlugins.map(p => p.teardown()));
  }
}

// plugins/transformers/normalize-timestamps.ts — a transformer plugin

import { PipelinePlugin } from '../../core/plugin.interface';

export class NormalizeTimestampsPlugin implements PipelinePlugin {
  readonly name = 'normalize-timestamps';
  readonly type = 'transformer' as const;
  private timezone = 'UTC';

  async initialize(config: Record<string, unknown>): Promise<void> {
    this.timezone = (config.timezone as string) ?? 'UTC';
  }

  async process(record: Record<string, unknown>): Promise<Record<string, unknown>> {
    const result = { ...record };
    for (const [key, value] of Object.entries(result)) {
      if (typeof value === 'string' && /^\d{4}-\d{2}-\d{2}/.test(value)) {
        result[key] = new Date(value).toISOString();
      }
    }
    return result;
  }

  async teardown(): Promise<void> {
    // Nothing to clean up
  }
}

// plugins/writers/postgres-writer.ts — a writer plugin

import { Pool } from 'pg';
import { PipelinePlugin } from '../../core/plugin.interface';

export class PostgresWriterPlugin implements PipelinePlugin {
  readonly name = 'postgres-writer';
  readonly type = 'writer' as const;
  private pool!: Pool;
  private tableName = 'pipeline_records';

  async initialize(config: Record<string, unknown>): Promise<void> {
    this.pool = new Pool({ connectionString: config.dbUrl as string });
    this.tableName = (config.tableName as string) ?? 'pipeline_records';
  }

  async process(record: Record<string, unknown>): Promise<Record<string, unknown>> {
    const keys = Object.keys(record);
    const values = Object.values(record);
    const placeholders = keys.map((_, i) => `$${i + 1}`).join(', ');

    await this.pool.query(
      `INSERT INTO ${this.tableName} (${keys.join(', ')}) VALUES (${placeholders})`,
      values
    );
    return record;
  }

  async teardown(): Promise<void> {
    await this.pool.end();
  }
}

// main.ts — wiring the core with plugins

import { PipelineCore } from './core/pipeline-core';
import { NormalizeTimestampsPlugin } from './plugins/transformers/normalize-timestamps';
import { PostgresWriterPlugin } from './plugins/writers/postgres-writer';
import { S3WriterPlugin } from './plugins/writers/s3-writer';

async function main() {
  const core = new PipelineCore();

  const timestampPlugin = new NormalizeTimestampsPlugin();
  await timestampPlugin.initialize({ timezone: 'Asia/Bangkok' });

  const postgresPlugin = new PostgresWriterPlugin();
  await postgresPlugin.initialize({
    dbUrl: process.env.DATABASE_URL,
    tableName: 'sensor_events',
  });

  const s3Plugin = new S3WriterPlugin();
  await s3Plugin.initialize({
    bucket: process.env.S3_BUCKET,
    prefix: 'pipeline/',
  });

  core.register(timestampPlugin);   // transformer
  core.register(postgresPlugin);    // writer
  core.register(s3Plugin);          // writer

  const records = await fetchRecordsFromSource();
  await core.process(records);
  await core.shutdown();
}

main().catch(console.error);

To add a new output target (Elasticsearch, webhook, BigQuery), I write a new plugin implementing PipelinePlugin. I do not touch PipelineCore or any existing plugin.

---

Real-World Examples

System	Core	Plugins
VS Code	Editor, layout, extension API	Language servers, debuggers, themes, formatters
webpack	Module bundler core	Loaders, plugins for transforms and outputs
Eclipse IDE	Platform, OSGi runtime	Language tools, debuggers, build integrations
Jenkins	Build orchestrator	SCM integrations, build steps, notifications
ESLint	Linting engine	Rules, plugins, formatters

All of these follow the same principle: stable core + extensible plugin system. Adding functionality does not require modifying the core.

---

When to Use Microkernel

• The system needs to be extended by different teams or third parties without modifying the central codebase

• Features have very different lifecycles — some parts change weekly, others are stable for years

• The system is a platform, not just an application — it needs to support use cases not known at build time

• Plugin isolation is important — one plugin breaking should not bring down the core

• A CLI tool or development tool that needs to be composable (think prettier, eslint, babel)

---

Challenges

• Plugin contract stability. Once a plugin interface is public, changing it breaks all plugins. Design the interface carefully and version it explicitly.

• Plugin discovery and loading in production. Dynamic plugin loading can introduce security risks — ensure plugins come from trusted sources.

• Inter-plugin dependencies. When plugins need to communicate with each other, the architecture gets complex. Try to route all communication through the core.

• Error isolation. A misbehaving plugin should not crash the core. Consider sandboxing or error boundaries around plugin execution.

---

Lessons Learned

• Start with the minimum viable core. I have made the mistake of over-engineering the core with features that "plugins will probably need." Ship the minimal core first; extend it only when a concrete plugin requires it.

• The plugin interface is a public API. Treat it with the same care as a public REST API — document it, version it, do not break it.

• Plugins should be independently deployable. If adding a plugin requires rebuilding the core, the boundary is not clean enough.

• TypeScript interfaces are ideal for defining plugin contracts. Strong typing ensures plugins implement the full contract and gives IDE support to plugin authors.