Multi-Backend Integration

Why I Send Telemetry to 5 Different Backends

My production setup sends telemetry to:

1. Jaeger (7 days) - Distributed tracing and debugging

2. Prometheus (30 days) - Metrics and alerting

3. AWS CloudWatch (365 days) - Compliance and auditing

4. Datadog (90 days) - Team collaboration and dashboards

5. New Relic (30 days) - APM and user experience monitoring

Each serves a specific purpose. Here's how I integrated them all without drowning in complexity.

Sending to Multiple Backends

Configuration Pattern

import { NodeSDK } from '@opentelemetry/sdk-node';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics';
import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http';
import { PrometheusExporter } from '@opentelemetry/exporter-prometheus';

// Multiple trace exporters
const jaegerExporter = new OTLPTraceExporter({
  url: 'http://jaeger:4318/v1/traces',
});

const datadogExporter = new OTLPTraceExporter({
  url: 'https://trace.agent.datadoghq.com/v1/traces',
  headers: {
    'DD-API-KEY': process.env.DD_API_KEY!,
  },
});

const newRelicExporter = new OTLPTraceExporter({
  url: 'https://otlp.nr-data.net/v1/traces',
  headers: {
    'api-key': process.env.NEW_RELIC_LICENSE_KEY!,
  },
});

// Multiple metric exporters
const prometheusExporter = new PrometheusExporter({
  port: 9464,
  endpoint: '/metrics',
});

const cloudwatchExporter = new OTLPMetricExporter({
  url: 'http://aws-otel-collector:4318/v1/metrics',
});

const sdk = new NodeSDK({
  // Send traces to multiple backends
  spanProcessors: [
    new BatchSpanProcessor(jaegerExporter),
    new BatchSpanProcessor(datadogExporter),
    new BatchSpanProcessor(newRelicExporter),
  ],
  
  // Metrics to Prometheus + CloudWatch
  metricReader: new PeriodicExportingMetricReader({
    exporter: cloudwatchExporter,
    exportIntervalMillis: 60000,
  }),
});

sdk.start();

Cloud Provider Integrations

AWS CloudWatch via ADOT Collector

AWS Distro for OpenTelemetry (ADOT):

# ECS Task Definition
{
  "family": "order-service",
  "containerDefinitions": [
    {
      "name": "order-service",
      "image": "myregistry/order-service:latest",
      "environment": [
        {
          "name": "OTEL_EXPORTER_OTLP_ENDPOINT",
          "value": "http://localhost:4318"
        }
      ]
    },
    {
      "name": "aws-otel-collector",
      "image": "public.ecr.aws/aws-observability/aws-otel-collector:latest",
      "command": ["--config=/etc/ecs/otel-config.yaml"],
      "environment": [
        {
          "name": "AWS_REGION",
          "value": "us-east-1"
        }
      ]
    }
  ]
}

ADOT Config (otel-config.yaml):

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 60s

exporters:
  # CloudWatch Metrics
  awsemf:
    namespace: OrderService
    region: us-east-1
    dimension_rollup_option: NoDimensionRollup
  
  # X-Ray Traces
  awsxray:
    region: us-east-1

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [awsxray]
    
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [awsemf]

View in CloudWatch:

# View traces in X-Ray
aws xray get-trace-summaries --start-time 2026-01-08T00:00:00 --end-time 2026-01-08T23:59:59

# View metrics in CloudWatch
aws cloudwatch get-metric-statistics \
  --namespace OrderService \
  --metric-name http.server.request.duration \
  --start-time 2026-01-08T00:00:00Z \
  --end-time 2026-01-08T23:59:59Z \
  --period 300 \
  --statistics Average

Google Cloud Operations

npm install @google-cloud/opentelemetry-cloud-trace-exporter
npm install @google-cloud/opentelemetry-cloud-monitoring-exporter

import { TraceExporter } from '@google-cloud/opentelemetry-cloud-trace-exporter';
import { MetricExporter } from '@google-cloud/opentelemetry-cloud-monitoring-exporter';
import { NodeSDK } from '@opentelemetry/sdk-node';

const sdk = new NodeSDK({
  traceExporter: new TraceExporter({
    projectId: process.env.GOOGLE_CLOUD_PROJECT,
  }),
  
  metricReader: new PeriodicExportingMetricReader({
    exporter: new MetricExporter({
      projectId: process.env.GOOGLE_CLOUD_PROJECT,
    }),
  }),
});

sdk.start();

Azure Monitor (Application Insights)

npm install @azure/monitor-opentelemetry-exporter

import { AzureMonitorTraceExporter } from '@azure/monitor-opentelemetry-exporter';
import { AzureMonitorMetricExporter } from '@azure/monitor-opentelemetry-exporter';

const sdk = new NodeSDK({
  traceExporter: new AzureMonitorTraceExporter({
    connectionString: process.env.APPLICATIONINSIGHTS_CONNECTION_STRING,
  }),
  
  metricReader: new PeriodicExportingMetricReader({
    exporter: new AzureMonitorMetricExporter({
      connectionString: process.env.APPLICATIONINSIGHTS_CONNECTION_STRING,
    }),
  }),
});

sdk.start();

Commercial Platforms

Datadog

npm install @opentelemetry/exporter-trace-otlp-http

const datadogExporter = new OTLPTraceExporter({
  url: 'https://trace.agent.datadoghq.com/v1/traces',
  headers: {
    'DD-API-KEY': process.env.DD_API_KEY!,
  },
});

const sdk = new NodeSDK({
  spanProcessors: [
    new BatchSpanProcessor(datadogExporter),
  ],
});

sdk.start();

Kubernetes with Datadog Agent:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: datadog-agent
spec:
  template:
    spec:
      containers:
      - name: datadog-agent
        image: gcr.io/datadoghq/agent:latest
        env:
        - name: DD_API_KEY
          valueFrom:
            secretKeyRef:
              name: datadog-secret
              key: api-key
        - name: DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT
          value: "0.0.0.0:4317"
        - name: DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT
          value: "0.0.0.0:4318"
        ports:
        - containerPort: 4317
        - containerPort: 4318

New Relic

const newRelicExporter = new OTLPTraceExporter({
  url: 'https://otlp.nr-data.net/v1/traces',
  headers: {
    'api-key': process.env.NEW_RELIC_LICENSE_KEY!,
  },
});

const sdk = new NodeSDK({
  spanProcessors: [
    new BatchSpanProcessor(newRelicExporter),
  ],
});

sdk.start();

Honeycomb

const honeycombExporter = new OTLPTraceExporter({
  url: 'https://api.honeycomb.io/v1/traces',
  headers: {
    'x-honeycomb-team': process.env.HONEYCOMB_API_KEY!,
    'x-honeycomb-dataset': 'production',
  },
});

const sdk = new NodeSDK({
  spanProcessors: [
    new BatchSpanProcessor(honeycombExporter),
  ],
});

sdk.start();

Lightstep

const lightstepExporter = new OTLPTraceExporter({
  url: 'https://ingest.lightstep.com:443/api/v2/otel/trace',
  headers: {
    'lightstep-access-token': process.env.LIGHTSTEP_ACCESS_TOKEN!,
  },
});

Smart Routing with Collector

Send different telemetry to different backends:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 10s
  
  # Filter production-only traces
  filter/production:
    traces:
      resource:
        - 'resource.attributes["deployment.environment"] == "production"'
  
  # Filter errors only
  filter/errors:
    traces:
      span:
        - 'status.code == ERROR'

exporters:
  # Jaeger: All traces
  otlp/jaeger:
    endpoint: jaeger:4317
    tls:
      insecure: true
  
  # Datadog: Production only
  otlp/datadog:
    endpoint: trace.agent.datadoghq.com:443
    headers:
      DD-API-KEY: ${DD_API_KEY}
  
  # PagerDuty: Errors only (for alerting)
  otlp/pagerduty:
    endpoint: events.pagerduty.com:443
    headers:
      Authorization: Token ${PAGERDUTY_TOKEN}
  
  # CloudWatch: Long-term compliance
  awsxray:
    region: us-east-1

service:
  pipelines:
    # Pipeline 1: All traces to Jaeger
    traces/jaeger:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/jaeger]
    
    # Pipeline 2: Production traces to Datadog
    traces/datadog:
      receivers: [otlp]
      processors: [filter/production, batch]
      exporters: [otlp/datadog]
    
    # Pipeline 3: Errors to PagerDuty
    traces/alerts:
      receivers: [otlp]
      processors: [filter/errors, batch]
      exporters: [otlp/pagerduty]
    
    # Pipeline 4: All traces to CloudWatch (compliance)
    traces/compliance:
      receivers: [otlp]
      processors: [batch]
      exporters: [awsxray]

Cost Management

Sampling by Backend

Different sampling rates for different purposes:

processors:
  # Datadog: 1% (expensive)
  tail_sampling/datadog:
    decision_wait: 10s
    policies:
      - name: error-policy
        type: status_code
        status_code:
          status_codes: [ERROR]
      - name: probabilistic
        type: probabilistic
        probabilistic:
          sampling_percentage: 1
  
  # Jaeger: 10% (cheaper, internal)
  tail_sampling/jaeger:
    decision_wait: 10s
    policies:
      - name: error-policy
        type: status_code
        status_code:
          status_codes: [ERROR]
      - name: probabilistic
        type: probabilistic
        probabilistic:
          sampling_percentage: 10

service:
  pipelines:
    traces/datadog:
      receivers: [otlp]
      processors: [tail_sampling/datadog, batch]
      exporters: [otlp/datadog]
    
    traces/jaeger:
      receivers: [otlp]
      processors: [tail_sampling/jaeger, batch]
      exporters: [otlp/jaeger]

Monthly Cost Comparison

At 10,000 req/s with different sampling rates:

Backend	Sampling	Data/Month	Cost/Month
Jaeger (self-hosted)	10%	390 GB	$40 (EC2 + storage)
Datadog	1%	39 GB	$350
New Relic	1%	39 GB	$280
CloudWatch	100% errors	20 GB	$50
Total	-	488 GB	$720

Without smart sampling: Would be $3,500/month!

Migration Strategies

Gradual Migration

Migrate from proprietary agent to OpenTelemetry:

// Phase 1: Dual instrumentation (Datadog + OTel)
import ddTrace from 'dd-trace';
import { NodeSDK } from '@opentelemetry/sdk-node';

// Keep existing Datadog
ddTrace.init();

// Add OpenTelemetry
const sdk = new NodeSDK({
  spanProcessors: [
    new BatchSpanProcessor(
      new OTLPTraceExporter({
        url: 'http://otel-collector:4318/v1/traces',
      })
    ),
  ],
});

sdk.start();

Phase 2: Route OTel to Datadog (via Collector)

exporters:
  otlp/datadog:
    endpoint: trace.agent.datadoghq.com:443
    headers:
      DD-API-KEY: ${DD_API_KEY}

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/datadog]

Phase 3: Remove Datadog agent, use OTel exclusively

Feature Comparison

Feature	Jaeger	Datadog	New Relic	Honeycomb
OpenTelemetry Support	Native	OTLP	OTLP	OTLP
Trace Sampling	Manual	Automatic	Automatic	Automatic
Custom Dashboards	Limited	Excellent	Excellent	Excellent
Alerting	Limited	Excellent	Excellent	Good
APM	No	Yes	Yes	No
Cost (per GB)	$0.10	$9	$7	$5
Self-hosted	Yes	No	No	No
Data Retention	7 days	15 days	8 days	60 days

Best Practices

1. Use Collector for routing - Don't configure backends in app code

2. Sample differently per backend - Expensive platforms get 1%, self-hosted get 10%

3. Standardize on OTLP - Avoid proprietary formats

4. Monitor export failures - Critical backends shouldn't fail silently

5. Test in staging first - Especially with commercial platforms

6. Set budget alerts - Cloud observability costs can spike

7. Archive to S3 - Long-term compliance data doesn't need expensive backends

Real Production Setup

Here's what I actually run:

# Collector configuration
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 10s
  
  # 1% for Datadog
  probabilistic/datadog:
    sampling_percentage: 1
  
  # 10% for Jaeger
  probabilistic/jaeger:
    sampling_percentage: 10
  
  # 100% errors
  filter/errors:
    traces:
      span:
        - 'status.code == ERROR'

exporters:
  # Internal debugging: Jaeger (10%, 7 days)
  otlp/jaeger:
    endpoint: jaeger:4317
  
  # Team collaboration: Datadog (1%, 15 days)
  otlp/datadog:
    endpoint: trace.agent.datadoghq.com:443
    headers:
      DD-API-KEY: ${DD_API_KEY}
  
  # Compliance: CloudWatch (errors only, 365 days)
  awsxray:
    region: us-east-1

service:
  pipelines:
    # Jaeger: 10% sampling
    traces/jaeger:
      receivers: [otlp]
      processors: [probabilistic/jaeger, batch]
      exporters: [otlp/jaeger]
    
    # Datadog: 1% sampling
    traces/datadog:
      receivers: [otlp]
      processors: [probabilistic/datadog, batch]
      exporters: [otlp/datadog]
    
    # CloudWatch: Errors only
    traces/cloudwatch:
      receivers: [otlp]
      processors: [filter/errors, batch]
      exporters: [awsxray]

Result:

• Jaeger: Fast debugging, 10% sample, $40/month

• Datadog: Team dashboards, 1% sample, $350/month

• CloudWatch: Compliance, errors only, $50/month

• Total: $440/month for full observability

What's Next

You've reached the end of the OpenTelemetry 101 series! Continue to Observability-Driven Development for the final chapter on building systems with observability in mind from day one.

---

Previous: ← Production Deployment | Next: Observability-Driven Development →

Vendor-neutral observability gives you freedom to choose the best tools.