WDP Part 9b: Wire Protocol Specification

HTTP transmission protocol for compact WDP diagnostics

Abstract

This document defines the wire protocol for transmitting WDP diagnostics over HTTP. It specifies how servers encode diagnostics in responses and how clients parse and expand them using catalogs. Part 9b focuses on runtime communication, while Part 9a covers catalog file format and Part 9c provides implementation guidance.

Key Points

Compact IDs as JSON keys for efficient lookup (CompactID or CombinedID format)
"wd" wrapper when mixing diagnostics with application data
Field placeholders use {{field}} syntax
Supports all severity types (E, W, C, I, H)
Wire protocol NEVER includes namespace strings (namespace-free)
Efficient wire format with minimal transmission overhead

1. Introduction

1.1 Purpose

The WDP wire protocol defines how diagnostics are transmitted over HTTP in a compact, efficient format. Servers send compact IDs and field values; clients expand them using pre-downloaded catalogs.

Key Benefits

Efficient transmission with compact wire format
Offline expansion - clients expand diagnostics locally
Language-agnostic - same compact IDs work for all languages
O(1) lookup - compact IDs as keys enable direct catalog access

1.2 Design Principles

Compact IDs as Keys - Direct JSON key-based access, no "h" field needed
Minimal Wrappers - Use "wd" only when necessary to avoid conflicts
Severity Agnostic - Same format for errors, warnings, info, help, critical
Natural Multi-Diagnostic - Single format works for 1 or N diagnostics
Flexible - Supports both standalone and wrapped formats
Namespace-Free - Wire protocol NEVER includes namespace strings or metadata

2. Compact Diagnostic Response Format

2.1 Standalone Diagnostics

When diagnostics are the only content in the response, use compact IDs as top-level keys:

{
  "<compact_id>": {
    "f": {
      "<field_name>": "<field_value>"
    }
  }
}

Field	Type	Required	Description
	key	Yes	CompactID (5 chars) or CombinedID (11 chars with hyphen)
f	object	No	Field values for message interpolation
pii	object	No	PII data with version and data fields

ID Format Patterns

Format	Pattern	Length	Use Case
CompactID	^[A-Za-z0-9]{5}$	5 chars	Single-namespace contexts
CombinedID	^[A-Za-z0-9]{5}-[A-Za-z0-9]{5}$	11 chars	Multi-namespace contexts

Important: Wire protocol NEVER includes namespace as a separate field. The namespace information is encoded in the ID format itself (via the namespace hash in CombinedID).

{
  "xY9Kp": {
    "f": {
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

{
  "jGKFp": {}
}

Note: When a diagnostic has no fields, use an empty object {} as the value. Do NOT use null.

2.2 Mixed with Application Data

When response contains both application data and diagnostics, use "wd" wrapper:

{
  "data": { ... },
  "wd": {
    "<compact_id>": {
      "f": { ... }
    }
  }
}

Rationale:

"wd" = "Waddling Diagnostic" or "WDP"
2 characters (vs 11 for "diagnostics")
Unambiguous prefix prevents key conflicts
Consistent with WDP's compact philosophy

{
  "order_id": "ORD-12345",
  "total": 99.99,
  "status": "created",
  "wd": {
    "wN4Qm": {
      "f": {
        "quota_remaining": "10"
      }
    }
  }
}

2.3 Multiple Diagnostics

Multiple diagnostics use the same format (multiple keys):

{
  "mN3Yr": {
    "f": {
      "field": "password",
      "min_length": 8
    }
  },
  "aG8eT": {
    "f": {
      "field": "email"
    }
  },
  "pL2Xk": {}
}

{
  "success": false,
  "wd": {
    "mN3Yr": {"f": {"field": "password", "min_length": 8}},
    "aG8eT": {"f": {"field": "email"}}
  }
}

2.4 Compact ID Format Semantics

CompactID Format (Single-Namespace)

Pattern: ^[A-Za-z0-9]{5}$
Length: 5 characters
Example: "81E9g", "xY9Kp"

Used by:

Single-namespace services
Mobile/web apps with one backend
Individual microservices
IoT devices (most cases)

CombinedID Format (Multi-Namespace)

Pattern: ^[A-Za-z0-9]{5}-[A-Za-z0-9]{5}$
Length: 11 characters
Format: <NamespaceHash>-<CompactID>
Example: "h4tYw2-81E9g", "k9Px3a-xY9Kp"

Used by:

Edge gateways aggregating multiple devices
Monitoring dashboards showing multiple services
Log aggregators collecting from multiple sources
API gateways routing to multiple backends

Server Decision Tree
function chooseIdFormat(diagnostics: Diagnostic[]): 'compact' | 'combined' {
  // If single namespace, use CompactID
  if (allFromSameNamespace(diagnostics)) {
    return 'compact';
  }
  
  // If multiple namespaces, MUST use CombinedID
  return 'combined';
}

Server MAY Always Use CombinedID: Servers MAY choose to always send CombinedID format for consistency, even when serving single-namespace responses.

3. HTTP Response Patterns

3.1 Pattern 1: Error (Standalone)

Use Case: Operation failed, diagnostics are the only content

HTTP/1.1 401 Unauthorized
Content-Type: application/json

{
  "xY9Kp": {
    "f": {
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

Error: Token expired at 2024-01-15T10:30:00Z

3.2 Pattern 2: Multiple Errors (Validation)

Use Case: Multiple validation failures

HTTP/1.1 422 Unprocessable Entity
Content-Type: application/json

{
  "mN3Yr": {
    "f": {
      "field": "password",
      "min_length": 8
    }
  },
  "aG8eT": {
    "f": {
      "field": "email"
    }
  }
}

Error: Password must be at least 8 characters
Error: Email address is required

3.3 Pattern 3: Warning with Success Data

Use Case: Operation succeeded but with warnings

HTTP/1.1 200 OK
Content-Type: application/json

{
  "data": {
    "id": "12345",
    "uploaded": true
  },
  "wd": {
    "wN4Qm": {
      "f": {
        "quota_used": "850",
        "quota_limit": "1000",
        "quota_percent": "85"
      }
    }
  }
}

✓ Upload successful
⚠ Warning: Storage quota at 850/1000 MB (85%)
  💡 Consider upgrading plan

3.4 Pattern 4: Critical System Alert

Use Case: Critical system-level diagnostic

HTTP/1.1 200 OK
Content-Type: application/json

{
  "status": "operational",
  "wd": {
    "cP9Wm": {
      "f": {
        "usage": "95",
        "mount_point": "/var/log"
      }
    }
  }
}

🔴 CRITICAL: Disk usage at 95% on /var/log
   💡 Free up disk space immediately

3.5 Pattern 5: Help/Info Messages

Use Case: Informational messages, guidance

HTTP/1.1 200 OK
Content-Type: application/json

{
  "requests": 990,
  "limit": 1000,
  "wd": {
    "hK3Qn": {
      "f": {
        "remaining": "10"
      }
    }
  }
}

ℹ️ Help: Rate limit: 10 requests remaining in this window
  💡 Implement exponential backoff

3.6 Pattern 6: Optional Header

Use Case: Fast filtering in logs/monitoring without parsing body

HTTP/1.1 401 Unauthorized
X-WDP-Diagnostic: xY9Kp
Content-Type: application/json

{
  "xY9Kp": {
    "f": {
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

HTTP/1.1 422 Unprocessable Entity
X-WDP-Diagnostic: mN3Yr,aG8eT,pL2Xk
Content-Type: application/json

{
  "mN3Yr": {"f": {...}},
  "aG8eT": {"f": {...}},
  "pL2Xk": {}
}

3.7 Severity-Specific HTTP Status Codes

Severity	HTTP Status	Example
E (Error)	4xx, 5xx	400, 401, 403, 404, 422, 500
W (Warning)	200, 201, 202	Success with warnings
C (Critical)	200, 500, 503	May succeed but critical alert
I (Info)	200, 201, 202	Success with informational message
H (Help)	200, 429	Guidance, rate limits

Note: These are recommendations. HTTP status code should reflect operation success/failure, not diagnostic severity.

4. Field Interpolation

4.1 Placeholder Syntax

Field placeholders in catalog messages use double curly brace syntax with support for namespaced PII fields:

"Token expired at {{timestamp}}"
"User {{user_id}} exceeded quota {{quota_limit}}"
"Login failed for {{pii/email}} at {{timestamp}}"
"Card ending in {{pii/last4}} declined"
"User {{pii/email:masked}} from {{pii/ip}} exceeded quota"

Placeholder Types

Type	Syntax	Source
Regular	{{field_name}}	f (fields) object
PII	{{pii/field_name}}	pii.data object
PII Masked	{{pii/field_name:masked}}	pii.data (pre-redacted)

Syntax Rules

Placeholders MUST be enclosed in {{ and }}
Regular field names MUST match regex: [a-zA-Z_][a-zA-Z0-9_]*
PII placeholders MUST use prefix pii/ followed by field name
PII placeholders MAY include optional :masked suffix
Single braces {field} are NOT valid
Placeholders MUST NOT be nested

4.2 Interpolation Process

// Step 1 - Catalog Message:
{ "message": "User {{user_id}} exceeded quota {{quota_limit}}" }

// Step 2 - Wire Protocol:
{ "f": { "user_id": "12345", "quota_limit": "1000" } }

// Step 3 - Result:
"User 12345 exceeded quota 1000"

// Step 1 - Catalog Message:
{ "message": "Login failed for {{pii/email}} at {{timestamp}}" }

// Step 2 - Wire Protocol:
{
  "f": { "timestamp": "2024-01-15T10:30:00Z" },
  "pii": {
    "v": 1,
    "data": { "email": "john@example.com" }
  }
}

// Step 3 - With PII access:
"Login failed for john@example.com at 2024-01-15T10:30:00Z"

// Step 3 - Without PII access:
"Login failed for [REDACTED] at 2024-01-15T10:30:00Z"

4.3 Missing Fields

If a field value is not provided in the wire protocol:

Option 1: Keep placeholder (RECOMMENDED) - "Token expired at {{timestamp}}"
Option 2: Use default marker - "Token expired at <missing>"
Option 3: Use empty string - "Token expired at "

4.4 PII Placeholder Semantics

PII placeholders support optional format suffixes to indicate redaction responsibility:

{{pii/field}}         → Defaults to :masked (safe default)
{{pii/field:raw}}     → Raw PII (receiver handles security)
{{pii/field:masked}}  → Pre-redacted PII (explicit, same as default)

4.4.1 Default Behavior (Masked)

Defaults to masked/redacted behavior
Sender should pre-redact the value
Receiver displays value as-is
Safe for untrusted recipients

4.4.2 Raw PII (Explicit Opt-In)

Sender transmits actual PII value
Receiver MUST handle securely
Use for trusted communication channels
Requires encrypted transport (TLS recommended)

4.5 PII Field Guidelines

Use {{pii/field}} (raw PII) for:

Email addresses, names, phone numbers
Physical addresses, IP addresses
Location data, device identifiers
Payment information, health information
Government IDs (SSN, passport numbers)

Use {{field}} (non-PII) for:

Anonymized user IDs (usr_123)
Timestamps, counts, quotas, percentages
System metadata, error codes
Feature flags, technical metrics

5. Client Expansion Process

5.1 Overview

Client receives compact diagnostic(s)
Extract compact IDs from response
Look up each ID in local catalog
Interpolate fields into message
Display with appropriate severity styling

5.2 Step-by-Step Example

// HTTP/1.1 401 Unauthorized
{
  "xY9Kp": {
    "f": {
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

const response = await fetch('/api/resource');
const body = await response.json();

// Extract compact IDs (keys matching 5-char Base62 pattern)
const compactIds = Object.keys(body).filter(
  key => /^[0-9A-Za-z]{5}$/.test(key)
);
// compactIds = ["xY9Kp"]

const catalog = await loadCatalog();
const entry = catalog.diags["xY9Kp"];

// entry = {
//   "code": "E.AUTH.TOKEN.EXPIRED",
//   "severity": "E",
//   "message": "Token expired at {{timestamp}}",
//   "description": "The JWT token has exceeded its TTL.",
//   "hints": ["Use /auth/refresh endpoint"]
// }

let message = entry.message;
const fields = body["xY9Kp"].f || {};

for (const [field, value] of Object.entries(fields)) {
  message = message.replace(`{{${field}}}`, value);
}

// message = "Token expired at 2024-01-15T10:30:00Z"

const severityLabels = {
  'E': { icon: '❌', label: 'Error', color: 'red' },
  'W': { icon: '⚠️', label: 'Warning', color: 'yellow' },
  'C': { icon: '🔴', label: 'Critical', color: 'red' },
  'I': { icon: 'ℹ️', label: 'Info', color: 'blue' },
  'H': { icon: '💡', label: 'Help', color: 'green' }
};

const s = severityLabels[entry.severity];
console.log(`${s.icon} ${s.label}: ${message}`);

// Output:
// ❌ Error: Token expired at 2024-01-15T10:30:00Z
//   💡 Use /auth/refresh endpoint

5.3 Handling Unknown Compact IDs

Javascript
function expandDiagnostic(compactId, data, catalog) {
  const entry = catalog.diags[compactId];
  
  if (!entry) {
    return {
      compactId,
      code: 'UNKNOWN',
      severity: 'E',
      message: `Unknown diagnostic: ${compactId}`,
      description: 'This diagnostic is not in the catalog. Update your catalog or contact support.'
    };
  }
  
  // ... normal expansion
}

5.4 Handling Mixed Responses

Javascript
function extractDiagnostics(response) {
  // Check for "wd" wrapper
  if (response.wd && typeof response.wd === 'object') {
    return response.wd;
  }
  
  // Check for top-level compact IDs (5-char) or combined IDs (11-char)
  const diagnostics = {};
  const idRegex = /^[A-Za-z0-9]{5}(-[A-Za-z0-9]{5})?$/;
  for (const [key, value] of Object.entries(response)) {
    if (idRegex.test(key)) {
      diagnostics[key] = value;
    }
  }
  
  return Object.keys(diagnostics).length > 0 ? diagnostics : null;
}

6. Namespace Context and Wire Protocol

6.1 Core Principle

Namespace information is NEVER transmitted as a separate field in the wire protocol.

Why?

Efficiency: Compact hash representation instead of full namespace string
Information hiding: Don't leak internal service names
Stability: Rename services without breaking clients
Simplicity: One ID, not ID + namespace

6.2 How Clients Know the Namespace

Single-namespace clients:

Already know they're talking to one service
Catalog metadata provides namespace name if needed
No ambiguity possible

// Wire: Just compact ID
{"81E9g": {"f": {...}}}

// Catalog optionally documents namespace
{
  "namespace": "auth_service",
  "diags": {"81E9g": {...}}
}

Multi-namespace clients (gateways, monitoring):

Use combined IDs that encode namespace hash
Catalog's namespaces index maps hash → name
Enable reverse lookup for display purposes

// Wire: Combined ID encodes namespace
{"h4tYw2-81E9g": {"f": {...}}}

// Catalog provides reverse lookup
{
  "namespaces": {"auth_service": "h4tYw2"},
  "diags": {"h4tYw2-81E9g": {...}}
}

6.3 Namespace Attribution Example

Typescript
function displayDiagnostic(key: string, data: any, catalog: Catalog) {
  const entry = catalog.diags[key];
  
  // Expand message
  const message = interpolate(entry.message, data.f);
  
  // Optional: Attribute to namespace
  if (key.includes('-') && catalog.namespaces) {
    const nsHash = key.split('-')[0];
    
    // Reverse lookup
    for (const [name, hash] of Object.entries(catalog.namespaces)) {
      if (hash === nsHash) {
        console.log(`Error from ${name}: ${message}`);
        return;
      }
    }
    
    // Fallback: Show hash if name not in index
    console.log(`Error from ${nsHash}: ${message}`);
  } else {
    // Single-namespace or no attribution needed
    console.log(`Error: ${message}`);
  }
}

6.4 Privacy-Preserving Mode

When catalog omits namespaces index:

Client can still:

✅ Lookup diagnostic by combined ID
✅ Display error message
✅ Show error code
✅ Function completely

Client cannot:

❌ Display human-readable namespace name (shows hash instead)
❌ Know internal service architecture

This is intentional - supports security-sensitive environments where exposing service names to clients is undesirable.

7. Catalog-Wire Format Compatibility

7.1 Over-Qualified IDs (CombinedID → Single-Namespace Catalog)

Q: What if wire protocol uses CombinedID but catalog is single-namespace?

This scenario can legitimately occur when:

Backend always uses fully-qualified IDs (defensive programming)
Gateway aggregates responses from multiple single-namespace backends
Client receives responses from multiple backend versions

Client Behavior (RECOMMENDED)
function lookupDiag(wireKey: string, catalog: Catalog): DiagEntry | null {
  // Step 1: Try direct lookup (covers matching formats)
  if (catalog.diags[wireKey]) {
    return catalog.diags[wireKey];
  }
  
  // Step 2: If wire format is CombinedID but catalog is single-namespace
  if (wireKey.includes('-')) {
    const [nsHash, compactId] = wireKey.split('-');
    
    // Optional: Verify namespace hash matches (if available)
    if (catalog.namespace_hash && catalog.namespace_hash !== nsHash) {
      console.warn(
        `Namespace mismatch: wire has ${nsHash}, catalog has ${catalog.namespace_hash}`
      );
    }
    
    // Fallback to CompactID lookup
    return catalog.diags[compactId] || null;
  }
  
  // Not found
  return null;
}

7.2 Under-Qualified IDs (CompactID → Aggregated Catalog)

Q: What if wire protocol uses CompactID but catalog is aggregated?

This is an error condition that SHOULD be rejected.

Client Behavior (Strict Rejection)
function lookupDiag(wireKey: string, catalog: Catalog): DiagEntry | null {
  const isAggregated = Object.keys(catalog.diags)[0]?.includes('-');
  
  // If aggregated catalog, require CombinedID format
  if (isAggregated && !wireKey.includes('-')) {
    throw new ProtocolError(
      `Protocol violation: Aggregated catalog requires CombinedID format, got '${wireKey}'`
    );
  }
  
  return catalog.diags[wireKey] || null;
}

7.3 Summary

Scenario	Catalog Type	Wire Format	Behavior
Over-qualified	Single-namespace	CombinedID	Graceful fallback
Under-qualified	Aggregated	CompactID	Strict rejection
Matched	Single-namespace	CompactID	Direct lookup O(1)
Matched	Aggregated	CombinedID	Direct lookup O(1)

Design Principle: The protocol favors correctness and explicitness. Over-qualification is tolerated (graceful degradation), but under-qualification is rejected (fail-fast).

8. Client Implementation

8.1 Client Parsing Helper

Javascript
function extractDiagnostics(response) {
  // Check for "wd" wrapper (diagnostics alongside application data)
  if (response.wd) {
    return response.wd;
  }
  
  // Extract top-level compact IDs (5-char) or combined IDs (11-char)
  const diagnostics = {};
  const idRegex = /^[A-Za-z0-9]{5}(-[A-Za-z0-9]{5})?$/;
  for (const [key, value] of Object.entries(response)) {
    if (idRegex.test(key)) {
      diagnostics[key] = value;
    }
  }
  return Object.keys(diagnostics).length > 0 ? diagnostics : null;
}

// Usage
const response = await fetch('/api/resource').then(r => r.json());
const diagnostics = extractDiagnostics(response);
await expandAndDisplay(diagnostics, catalog);

8.2 Version Detection

Use catalog version or custom header to detect support:

GET /api/resource
X-WDP-Version: 1.0

HTTP/1.1 200 OK
X-WDP-Version: 1.0

{
  "xY9Kp": {"f": {...}}
}

Appendix

A. Wire Format Benefits

Bandwidth Comparison

Format	Example Size	Reduction
Full Message (Old)	158 bytes	-
WDP Wire Format	71 bytes	55%

{
  "error": {
    "code": "E.AUTH.TOKEN.EXPIRED",
    "message": "Token expired at 2024-01-15T10:30:00Z",
    "description": "The JWT token has exceeded its TTL."
  }
}

{
  "xY9Kp": {
    "f": {
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

Multiple Diagnostics Comparison

Full Messages (Old): 245 bytes

WDP Wire Format (New): 93 bytes (62% reduction)

B. Complete Wire Protocol Examples

B.1 Error Example (E)

POST /api/login HTTP/1.1
Content-Type: application/json

{
  "email": "",
  "password": "123"
}

HTTP/1.1 422 Unprocessable Entity
X-WDP-Diagnostic: aG8eT,mN3Yr
Content-Type: application/json

{
  "aG8eT": {
    "f": { "field": "email" }
  },
  "mN3Yr": {
    "f": { "field": "password", "min_length": 8 }
  }
}

B.2 Warning Example (W)

HTTP/1.1 200 OK
Content-Type: application/json

{
  "file_id": "f-12345",
  "url": "https://cdn.example.com/files/12345",
  "wd": {
    "wN4Qm": {
      "f": {
        "quota_used": "850",
        "quota_limit": "1000",
        "quota_percent": "85"
      }
    }
  }
}

B.3 Critical Example (C)

HTTP/1.1 200 OK
X-WDP-Diagnostic: cP9Wm
Content-Type: application/json

{
  "status": "degraded",
  "wd": {
    "cP9Wm": {
      "f": {
        "usage": "95",
        "mount_point": "/var/log"
      }
    }
  }
}

B.4 Help Example (H)

HTTP/1.1 200 OK
Content-Type: application/json

{
  "data": [...],
  "requests": 990,
  "limit": 1000,
  "wd": {
    "hK3Qn": {
      "f": { "remaining": "10" }
    }
  }
}