Extended Part9 of 12 (Main)

Sub-parts9a, 9b, 9c

Version0.1.0-draft

Last Updated2025-11-30

StatusDraft

TypeNORMATIVE (Extended)

WDP Part 9: Catalogs - Overview and Summary

Catalogs enable compact error transmission by mapping short hash identifiers to full diagnostic metadata, providing efficient communication for distributed systems, IoT devices, and mobile applications.

Abstract#

Part 9 defines the catalog system for the Waddling Diagnostic Protocol (WDP). Catalogs enable compact error transmission by mapping short hash identifiers to full diagnostic metadata, providing efficient communication for distributed systems, IoT devices, and mobile applications.

Part 9 is split into three focused specifications:

Part 9a: Catalog format (JSON structure and validation) — NORMATIVE
Part 9b: Wire protocol (compact transmission format) — NORMATIVE
Part 9c: Implementation guide (examples and best practices) — INFORMATIVE

This document provides a high-level overview of how these parts work together.

Conformance

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

1. What Are Catalogs?#

Problem: Sending full error details over the network is verbose and wasteful.

Verbose format:

{
  "error": {
    "code": "E.Auth.Token.001",
    "severity": "E",
    "message": "Token expired at 2024-01-15T10:30:00Z",
    "description": "The JWT token has exceeded its time-to-live.",
    "hints": ["Use /auth/refresh endpoint with refresh token"]
  }
}

Solution: Send compact hash + fields, client expands using catalog.

Compact:

{
  "xY9Kp": {
    "f": {"timestamp": "2024-01-15T10:30:00Z"}
  }
}

Catalogs provide:

Efficient transmission - Compact wire format
Offline capability - Clients cache catalogs locally
Multi-language support - Same hash, different language catalogs
Fast lookups - O(1) hash-based retrieval
Namespace support - Single or aggregated catalogs for different contexts

1.1 Namespaces in Catalogs

Motivation

Catalogs may be scoped to a single service/namespace or aggregate multiple namespaces for tooling and distribution. Wire protocol stays independent; catalogs carry context for humans and offline resolution.

Catalog Types

Single-Namespace Catalog (Default):

Used by most clients (mobile apps, web apps, individual services)
Keys in diags are simple compact IDs (e.g., 81E9g)
MAY omit namespace information entirely
MAY include optional namespace field for documentation

{
  "version": "1.0.0",
  "diags": {
    "81E9g": {"c": "E.Auth.Token.001", "m": "Token missing"}
  }
}

Aggregated Catalog (Multi-Namespace):

Used by gateways, monitoring tools, SDK bundles, log aggregators
Keys in diags are combined IDs (e.g., h4tYw2-81E9g) to avoid collisions
MAY include namespaces index for human readability
Can operate in privacy-preserving mode (hashes only, no names)

{
  "version": "1.0.0",
  "namespaces": {
    "auth_lib": "h4tYw2",
    "payment_lib": "k9Px3a"
  },
  "diags": {
    "h4tYw2-81E9g": {"c": "E.Auth.Token.001", "m": "Token missing"},
    "k9Px3a-00A1b": {"c": "E.Payment.Card.003", "m": "Card declined"}
  }
}

Field Summary

namespace (or ns): Optional. Present only if catalog is single-namespace and you want to name it.
namespace_hash (or nsh): Optional. Verifiable binding between named namespace and hashes.
namespaces (or nss): Optional. Only for aggregated catalogs; maps namespace names to hashes.

Key Identity Rules

Collisions and Identity:

If diags keys are compact IDs, catalog MUST be single-namespace (implicit or explicit)
If not single-namespace, keys MUST be combined IDs (<namespace_hash>-<compact_id>)
Combined ID format is stable even if human-readable namespace name changes

Wire Protocol Independence:

Wire protocol NEVER includes namespace as a separate field
Only compact IDs or combined IDs are transmitted
Namespace context is resolved client-side using the catalog

Security/Privacy Note

Avoid disclosing sensitive service names to untrusted audiences. Aggregated catalogs MAY omit the namespaces index and rely on hashes alone if name disclosure is undesirable.

Privacy-preserving aggregated catalog:

{
  "version": "1.0.0",
  "diags": {
    "h4tYw2-81E9g": {"c": "E.Auth.Token.001", "m": "Token missing"},
    "k9Px3a-00A1b": {"c": "E.Payment.Card.003", "m": "Card declined"}
  }
}

Clients can lookup and display diagnostics without knowing internal service names.

2. The Complete Flow#

The catalog system operates through a series of stages from build time to runtime:

Stage 1: Build Time - Generate Catalog (Part 9a + 9c)

Error Registry → Catalog Generator → catalog.json

{
  "version": "1.0.0",
  "diags": {
    "xY9Kp": {
      "code": "E.Auth.Token.001",
      "severity": "E",
      "message": "Token expired at {{timestamp}}",
      "description": "JWT token exceeded TTL",
      "hints": ["Use /auth/refresh endpoint"]
    }
  }
}

Stage 2: Distribution - Serve Catalog

CDN / Static File Server
https://cdn.example.com/wdp/catalog-v1.0.0.json

• Cacheable (long TTL)
• Version in URL
• Gzipped (further compression)

Stage 3: Runtime - Client Downloads & Caches (Part 9c)

Client (Browser/App):
  1. Download catalog on startup
  2. Cache in memory / localStorage
  3. Use for all subsequent error expansions

Stage 4: Runtime - Compact Transmission (Part 9b)

Server API Response:
{
  "success": false,
  "xY9Kp": {                    ← Compact ID
    "f": {                      ← Field values only
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

Efficient transmission with compact format

Stage 5: Runtime - Client Expansion (Part 9c)

Client expands using catalog:

const entry = catalog.diags["xY9Kp"];
const message = entry.message
  .replace("{{timestamp}}", fields.timestamp);

Result:
{
  "code": "E.Auth.Token.001",
  "severity": "E",
  "message": "Token expired at 2024-01-15T10:30:00Z",
  "description": "JWT token exceeded TTL",
  "hints": ["Use /auth/refresh endpoint"]
}

3. Part 9a: Catalog Format#

9a-CATALOG-FORMAT.md defines the JSON structure for catalog files.

What It Covers

1. Three Catalog Formats:

Full format - Complete metadata (development, debugging)
Compact format - Abbreviated keys (production, bandwidth-sensitive)
Minimal format - Only essential data (embedded systems, IoT)

2. Field Specifications:

3. Per-Diagnostic Fields:

4. Distribution Strategies:

Static file hosting (CDN) - RECOMMENDED
API endpoint (dynamic)
Bundled with application (offline-first)
Versioning and caching recommendations

Example (Full Format)

{
  "version": "1.0.0",
  "generated": "2024-01-15T10:30:00Z",
  "diags": {
    "xY9Kp": {
      "code": "E.Auth.Token.001",
      "severity": "E",
      "message": "Token expired at {{timestamp}}",
      "description": "The JWT token has exceeded its time-to-live.",
      "hints": [
        "Use /auth/refresh endpoint with refresh token",
        "Check token expiration time (exp claim)"
      ],
      "tags": ["auth", "security", "jwt"],
      "fields": ["timestamp"]
    }
  }
}

Example (Compact Format)

{
  "v": "1.0.0",
  "g": "2024-01-15T10:30:00Z",
  "wd": {
    "xY9Kp": {
      "c": "E.Auth.Token.001",
      "s": "E",
      "m": "Token expired at {{timestamp}}",
      "d": "JWT token exceeded TTL",
      "h": ["Use /auth/refresh endpoint"],
      "t": ["auth", "jwt"],
      "f": ["timestamp"]
    }
  }
}

See 9a-CATALOG-FORMAT.md for complete specification.

4. Part 9b: Wire Protocol#

9b-WIRE-PROTOCOL.md defines how to transmit compact diagnostics over the network.

What It Covers

1. Compact Diagnostic Response Format:

{
  "<compact_id>": {
    "f": {
      "<field_name>": <value>
    }
  }
}

2. Response Patterns:

Standalone diagnostics - Error-only responses
Mixed with application data - Diagnostics alongside data
Multiple diagnostics - Multiple errors at once
HTTP status codes - Mapping severity to status codes

3. Field Interpolation:

Placeholder syntax: {{field_name}}
Field value types (string, number, boolean)
Missing field handling

4. Client Expansion Process:

Step-by-step expansion algorithm
Handling unknown compact IDs
Extracting diagnostics from mixed responses

HTTP Response Examples

Pattern 1: Error (Standalone)

HTTP/1.1 401 Unauthorized
Content-Type: application/json

{
  "xY9Kp": {
    "f": {
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

Pattern 2: Warning with Success Data

HTTP/1.1 200 OK
Content-Type: application/json

{
  "data": {
    "user_id": 123,
    "quota_used": 850,
    "quota_limit": 1000
  },
  "wd": {
    "wN4Qm": {
      "f": {
        "quota_used": 850,
        "quota_limit": 1000
      }
    }
  }
}

Pattern 3: Multiple Errors (Validation)

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
  "mN3Yr": {
    "f": {
      "field": "password",
      "min_length": 8
    }
  },
  "aG8eT": {
    "f": {
      "field": "email"
    }
  }
}

See 9b-WIRE-PROTOCOL.md for complete specification.

5. Part 9c: Implementation Guide#

9c-IMPLEMENTATION.md provides practical implementation guidance with code examples.

What It Covers

1. Server-Side Catalog Generation:

Rust example using serde_json
Python example using dataclasses
TypeScript example for Node.js
Both full and compact format generation

2. Server-Side Diagnostic Responses:

Generating compact responses
Field interpolation
Error vs warning patterns
Mixed response handling

3. Client-Side Catalog Loading:

TypeScript/JavaScript implementation
Python implementation
Browser caching strategies (localStorage)
Catalog versioning and updates

4. Client-Side Expansion:

Expansion algorithm
Field substitution
Unknown compact ID handling
Multiple diagnostic expansion

5. Testing:

Catalog validation tests
Expansion tests
Integration tests
End-to-end examples

Server Example (Rust)

Rust
use serde_json::json;

fn generate_compact_response(
    compact_id: &str,
    fields: serde_json::Value
) -> serde_json::Value {
    json!({
        compact_id: {
            "f": fields
        }
    })
}

// Usage
let response = generate_compact_response(
    "xY9Kp",
    json!({"timestamp": "2024-01-15T10:30:00Z"})
);

Client Example (TypeScript)

Typescript
class WDPCatalog {
  private catalog: Catalog;

  async load(url: string): Promise<void> {
    const response = await fetch(url);
    this.catalog = await response.json();
  }

  expand(response: any): ExpandedDiagnostic[] {
    const results: ExpandedDiagnostic[] = [];
    
    for (const [compactId, data] of Object.entries(response)) {
      const entry = this.catalog.diags[compactId];
      if (!entry) continue;
      
      let message = entry.message;
      for (const [key, value] of Object.entries(data.f || {})) {
        message = message.replace(`{{${key}}}`, String(value));
      }
      
      results.push({
        code: entry.code,
        compact_id: compactId,
        severity: entry.severity,
        message: message,
        description: entry.description,
        hints: entry.hints
      });
    }
    
    return results;
  }
}

See 9c-IMPLEMENTATION.md for complete examples.

6. How the Parts Fit Together#

┌──────────────────────────────────────────────────────────┐
│                     Part 9a                              │
│              CATALOG FORMAT                              │
│                                                          │
│  Defines JSON structure:                                 │
│  • Full/compact/minimal formats                          │
│  • Field specifications                                  │
│  • Validation rules                                      │
│  • Distribution strategies                               │
└──────────────────────────────────────────────────────────┘
                         ↓ produces
┌──────────────────────────────────────────────────────────┐
│                   catalog.json                           │
│                                                          │
│  The actual catalog file distributed to clients          │
└──────────────────────────────────────────────────────────┘
                         ↓ used by
┌──────────────────────────────────────────────────────────┐
│                     Part 9b                              │
│               WIRE PROTOCOL                              │
│                                                          │
│  Defines transmission format:                            │
│  • Compact response structure                            │
│  • Field interpolation                                   │
│  • HTTP patterns                                         │
│  • Expansion algorithm                                   │
└──────────────────────────────────────────────────────────┘
                         ↓ implemented using
┌──────────────────────────────────────────────────────────┐
│                     Part 9c                              │
│            IMPLEMENTATION GUIDE                          │
│                                                          │
│  Provides code examples:                                 │
│  • Catalog generation (Rust, Python, TypeScript)         │
│  • Server-side response generation                       │
│  • Client-side loading and caching                       │
│  • Expansion logic with examples                         │
│  • Testing patterns                                      │
└──────────────────────────────────────────────────────────┘

Design Philosophy

Part 9a is WHAT (the catalog structure)
Part 9b is HOW (the transmission protocol)
Part 9c is IMPLEMENT (practical code examples)

All three work together to provide a complete catalog system.

7. Quick Start Guide#

Step 1: Generate Catalog (Part 9a + 9c)

# Build time
$ wdp-catalog-gen --input error-registry.json --output catalog.json

Step 2: Distribute Catalog (Part 9a)

# Upload to CDN
$ aws s3 cp catalog.json s3://cdn.example.com/wdp/catalog-v1.0.0.json

Step 3: Client Downloads Catalog (Part 9c)

const catalog = new WDPCatalog();
await catalog.load('https://cdn.example.com/wdp/catalog-v1.0.0.json');

Step 4: Server Sends Compact Response (Part 9b)

// Server
let response = json!({
    "xY9Kp": {
        "f": {
            "timestamp": "2024-01-15T10:30:00Z"
        }
    }
});

Step 5: Client Expands (Part 9c)

// Client
const diagnostics = catalog.expand(response);
console.log(diagnostics[0].message);
// → "Token expired at 2024-01-15T10:30:00Z"

8. When to Use Catalogs#

8.1 Catalog Type Selection

Use Single-Namespace Catalogs When:

Client talks to one backend/service
Mobile application with single API
Web application (SPA) with one backend
Individual microservice serving its own errors
IoT device (most cases - talks to one service)
Single library/SDK

Use Aggregated Catalogs When:

Monitoring dashboard showing multiple services
Edge gateway aggregating multiple devices/services
Log aggregator collecting from multiple sources
SDK bundle with multiple dependencies
Operations/observability tools
Multi-tenant platforms
API gateway/BFF routing to multiple backends

IoT-Specific Guidance

Simple IoT Devices (NO namespace):

Temperature Sensor → Gateway → Cloud
└─ Single-namespace catalog
└─ Keys: Simple compact IDs

Edge Gateways (YES namespace):

Sensor A ─┐
Sensor B ─┤
Sensor C ─┼→ Gateway → Cloud
Camera ───┤
Lock ─────┘
└─ Aggregated catalog needed
└─ Keys: Combined IDs

Industrial Hubs (YES namespace):

50+ Sensors → Industrial Gateway → Multiple Cloud Services
                                    (AWS IoT, Azure, SCADA)
└─ Aggregated catalog with namespace tracking
└─ Keys: Combined IDs for disambiguation

8.2 Use Catalogs When

Bandwidth is limited:

IoT devices with cellular connections
Mobile applications on metered networks
High-volume APIs with cost concerns
International users with slow connections

Client-side expansion is beneficial:

Multi-language support needed (localized catalogs)
Offline capability desired (cache catalogs)
Client-side error handling/formatting
Rich client-side error UI

Scale matters:

Many errors across distributed system
Frequent error transmission
Multiple services generating errors

8.3 Consider Alternatives When

Simple use cases:

Internal tools with fast networks
Low error volume
Single-language only
No offline requirements

Already using full responses:

REST APIs already sending full JSON
GraphQL with detailed errors
WebSocket with verbose messages

Team capacity:

Small team without time for catalog infrastructure
MVP/prototype phase
Simple monolithic application

Summary#

Part 9 provides a complete catalog system for WDP:

9a defines the catalog JSON format, validation, and namespace fields
9b defines the compact wire protocol for transmission (namespace-free)
9c provides implementation examples and best practices

Together, they enable:

Efficient transmission through compact wire format
Offline capability via catalog caching
Multi-language support with localized catalogs
Fast, efficient error handling in distributed systems
Single-namespace and aggregated catalog patterns
Privacy-preserving error transmission (hashes only)
Collision-free multi-service error aggregation

Catalog Types:

Single-namespace: Simple compact IDs, most clients (90%)
Aggregated: Combined IDs with optional names index, gateways/tools (10%)

Read the detailed specifications:

9a-CATALOG-FORMAT.md - Catalog structure and namespace fields
9b-WIRE-PROTOCOL.md - Transmission format (namespace-free wire protocol)
9c-IMPLEMENTATION.md - Code examples