DocumentWDP-9

TitleCatalogs Overview

Version0.1.0-draft

Last Updated2025-12-25

StatusDraft

CategoryStandards Track (Extended)

ObsoletesNone

UpdatesNone

WDP Part 9: Catalogs Overview

Overview of the catalog system for mapping compact hash identifiers to complete diagnostic metadata.

Abstract#

This document provides an overview of the catalog system for the Waddling Diagnostic Protocol (WDP). Catalogs constitute the mechanism by which compact hash identifiers are mapped to complete diagnostic metadata, enabling efficient communication for distributed systems, constrained devices, and mobile applications.

Part 9 is organized into three subordinate specifications:

Part 9a: Catalog Format Specification (JSON structure and validation) — NORMATIVE
Part 9b: Wire Protocol Specification (compact transmission format) — NORMATIVE
Part 9c: Implementation Guide (examples and best practices) — INFORMATIVE

This document provides a high-level overview of how these specifications interrelate.

Specification Navigation: Refer to STRUCTURE for an overview of all WDP specification documents.

Status of This Memo#

This document specifies a standards track protocol for the WDP community and requests discussion and suggestions for improvements. Distribution of this memo is unlimited.

Requirements Language#

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

1. Introduction#

1.1 Problem Statement#

Transmission of complete diagnostic information over network connections presents efficiency challenges:

Verbose Format (Inefficient):

{
  "error": {
    "code": "E.Auth.Token.001",
    "severity": "E",
    "message": "Token expired at 2024-01-15T10:30:00Z",
    "description": "The JWT token has exceeded its time-to-live.",
    "hints": ["Use /auth/refresh endpoint with refresh token"]
  }
}

Compact Format (Efficient):

{
  "xY9Kp": {
    "f": {"timestamp": "2024-01-15T10:30:00Z"}
  }
}

1.2 Solution Overview#

The WDP catalog system addresses this requirement through separation of concerns:

Build Time: Generate catalog mapping compact IDs to diagnostic metadata
Distribution: Serve catalog via CDN or static hosting
Runtime: Transmit compact IDs with field values only
Client-Side: Expand compact IDs using cached catalog

1.3 Catalog System Benefits#

The catalog system provides the following capabilities:

Efficient Transmission: Compact wire format reduces bandwidth consumption
Cross-System Resolution: Tools can resolve errors from any WDP-compliant service by caching catalogs, without needing source code or service-specific integrations
Multi-Language Support: Identical compact IDs across language-specific catalogs
Fast Lookups: O(1) hash-based retrieval
Namespace Support: Single-namespace and aggregated catalog patterns

1.4 Namespace Support in Catalogs#

1.4.1 Motivation#

Catalogs MAY be scoped to a single service or namespace, or MAY aggregate multiple namespaces for tooling and distribution purposes. The wire protocol remains independent of namespace context; catalogs provide context for human operators and offline resolution.

1.4.2 Catalog Types#

Single-Namespace Catalog (Default):

Single-namespace catalogs serve most client applications (mobile applications, web applications, individual services). Keys in the diags object are simple compact IDs.

{
  "version": "1.0.0",
  "diags": {
    "81E9g": {"c": "E.Auth.Token.001", "m": "Token missing"}
  }
}

Aggregated Catalog (Multi-Namespace):

Aggregated catalogs serve gateways, monitoring tools, SDK bundles, and log aggregators. Keys in the diags object are combined IDs to prevent collisions.

{
  "version": "1.0.0",
  "namespaces": {
    "auth_lib": "h4tYw2",
    "payment_lib": "k9Px3a"
  },
  "diags": {
    "h4tYw2-81E9g": {"c": "E.Auth.Token.001", "m": "Token missing"},
    "k9Px3a-00A1b": {"c": "E.Payment.Card.003", "m": "Card declined"}
  }
}

1.4.3 Field Summary#

Field	Compact	Purpose	Catalog Type
namespace	ns	Human-readable namespace name	Single-namespace only
namespace_hash	nsh	Verifiable namespace hash	Single-namespace only
namespaces	nss	Namespace name-to-hash index	Aggregated only

1.4.4 Key Identity Rules#

Collision Prevention:

If diags keys are compact IDs, the catalog MUST be single-namespace
If the catalog is not single-namespace, keys MUST be combined IDs (<namespace_hash>-<compact_id>)
Combined ID format remains stable regardless of namespace name changes

Wire Protocol Independence:

The wire protocol MUST NOT include namespace as a separate field
Only compact IDs or combined IDs are transmitted
Namespace context is resolved client-side using the catalog

1.4.5 Privacy Considerations#

Implementers SHOULD avoid disclosing internal service names to untrusted audiences. By omitting the namespaces index, an aggregated catalog can be publicly distributed without revealing the internal architecture or service names of the system, as the namespace hashes are opaque.

Privacy-Preserving Aggregated Catalog:

{
  "version": "1.0.0",
  "diags": {
    "h4tYw2-81E9g": {"c": "E.Auth.Token.001", "m": "Token missing"},
    "k9Px3a-00A1b": {"c": "E.Payment.Card.003", "m": "Card declined"}
  }
}

2. Catalog System Architecture#

2.1 Complete Flow#

┌────────────────────────────────────────────────────────────────┐
│ BUILD TIME: Generate Catalog (Part 9a + 9c)                    │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│  Error Registry → Catalog Generator → catalog.json             │
│                                                                │
│  {                                                             │
│    "version": "1.0.0",                                         │
│    "diags": {                                                  │
│      "xY9Kp": {                                                │
│        "code": "E.Auth.Token.001",                             │
│        "severity": "E",                                        │
│        "message": "Token expired at {{timestamp}}",            │
│        "description": "JWT token exceeded TTL",                │
│        "hints": ["Use /auth/refresh endpoint"]                 │
│      }                                                         │
│    }                                                           │
│  }                                                             │
│                                                                │
└────────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌────────────────────────────────────────────────────────────────┐
│ DISTRIBUTION: Serve Catalog                                    │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│  CDN / Static File Server                                      │
│  https://cdn.example.com/wdp/catalog-v1.0.0.json              │
│                                                                │
│  Properties:                                                   │
│  • Cacheable (extended TTL)                                    │
│  • Version encoded in URL                                      │
│  • Compressed (gzip)                                           │
│                                                                │
└────────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌────────────────────────────────────────────────────────────────┐
│ RUNTIME: Client Downloads and Caches (Part 9c)                 │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│  Client (Browser/Application):                                 │
│    1. Download catalog on initialization                       │
│    2. Cache in memory / persistent storage                     │
│    3. Use for all subsequent diagnostic expansions             │
│                                                                │
└────────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌────────────────────────────────────────────────────────────────┐
│ RUNTIME: Compact Transmission (Part 9b)                        │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│  Server API Response:                                          │
│  {                                                             │
│    "success": false,                                           │
│    "xY9Kp": {                    ← Compact ID                  │
│      "f": {                      ← Field values only           │
│        "timestamp": "2024-01-15T10:30:00Z"                     │
│      }                                                         │
│    }                                                           │
│  }                                                             │
│                                                                │
└────────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌────────────────────────────────────────────────────────────────┐
│ RUNTIME: Client Expansion (Part 9c)                            │
├────────────────────────────────────────────────────────────────┤
│                                                                │
│  Client expands using catalog:                                 │
│                                                                │
│  const entry = catalog.diags["xY9Kp"];                         │
│  const message = entry.message                                 │
│    .replace("{{timestamp}}", fields.timestamp);                │
│                                                                │
│  Result:                                                       │
│  {                                                             │
│    "code": "E.Auth.Token.001",                                 │
│    "severity": "E",                                            │
│    "message": "Token expired at 2024-01-15T10:30:00Z",         │
│    "description": "JWT token exceeded TTL",                    │
│    "hints": ["Use /auth/refresh endpoint"]                     │
│  }                                                             │
│                                                                │
└────────────────────────────────────────────────────────────────┘

3. Part 9a: Catalog Format#

Document: 9a-CATALOG-FORMAT.md

Part 9a specifies the JSON structure for catalog files.

3.1 Scope#

Catalog Format Variants:

Full Format: Complete metadata (development, debugging)
Compact Format: Abbreviated keys (production, bandwidth-sensitive)
Minimal Format: Essential data only (embedded systems, IoT)

Field Specifications:

version (REQUIRED) — Catalog version
generated (OPTIONAL) — Generation timestamp
diags (REQUIRED) — Diagnostic entries indexed by compact ID
Per-diagnostic: code, severity, message, description, hints, tags, fields

Validation:

Required fields verification
Compact ID format validation
Field placeholder validation
JSON schema for automated validation

Distribution:

Static file hosting (CDN) — RECOMMENDED
API endpoint (dynamic)
Bundled with application (offline-first)
Versioning and caching guidance

3.2 Example (Full Format)#

{
  "version": "1.0.0",
  "generated": "2024-01-15T10:30:00Z",
  "diags": {
    "xY9Kp": {
      "code": "E.Auth.Token.001",
      "severity": "E",
      "message": "Token expired at {{timestamp}}",
      "description": "The JWT token has exceeded its time-to-live.",
      "hints": [
        "Use /auth/refresh endpoint with refresh token",
        "Check token expiration time (exp claim)"
      ],
      "tags": ["auth", "security", "jwt"],
      "fields": ["timestamp"]
    }
  }
}

3.3 Example (Compact Format)#

{
  "v": "1.0.0",
  "g": "2024-01-15T10:30:00Z",
  "wd": {
    "xY9Kp": {
      "c": "E.Auth.Token.001",
      "s": "E",
      "m": "Token expired at {{timestamp}}",
      "d": "JWT token exceeded TTL",
      "h": ["Use /auth/refresh endpoint"],
      "t": ["auth", "jwt"],
      "f": ["timestamp"]
    }
  }
}

4. Part 9b: Wire Protocol#

Document: 9b-WIRE-PROTOCOL.md

Part 9b specifies the transmission format for compact diagnostics over network connections.

4.1 Scope#

Compact Diagnostic Response Format:

{
  "<compact_id>": {
    "f": {
      "<field_name>": <value>
    }
  }
}

Response Patterns:

Standalone diagnostics — Diagnostic-only responses
Mixed with application data — Diagnostics alongside data
Multiple diagnostics — Multiple diagnostics in single response
HTTP status code mapping — Severity to status code correlation

Field Interpolation:

Placeholder syntax: {{field_name}}
Field value types (string, number, boolean)
Missing field handling

Client Expansion Process:

Step-by-step expansion algorithm
Unknown compact ID handling
Diagnostic extraction from mixed responses

4.2 HTTP Response Examples#

Pattern 1: Error (Standalone)

HTTP/1.1 401 Unauthorized
Content-Type: application/json

{
  "xY9Kp": {
    "f": {
      "timestamp": "2024-01-15T10:30:00Z"
    }
  }
}

Pattern 2: Warning with Success Data

HTTP/1.1 200 OK
Content-Type: application/json

{
  "data": {
    "user_id": 123,
    "quota_used": 850,
    "quota_limit": 1000
  },
  "wd": {
    "wN4Qm": {
      "f": {
        "quota_used": 850,
        "quota_limit": 1000
      }
    }
  }
}

Pattern 3: Multiple Diagnostics (Validation)

HTTP/1.1 400 Bad Request
Content-Type: application/json

{
  "mN3Yr": {
    "f": {
      "field": "password",
      "min_length": 8
    }
  },
  "aG8eT": {
    "f": {
      "field": "email"
    }
  }
}

5. Part 9c: Implementation Guide#

Document: 9c-IMPLEMENTATION.md

Part 9c provides practical implementation guidance with code examples.

5.1 Scope#

Server-Side Catalog Generation:

Rust implementation using serde_json
Python implementation using dataclasses
TypeScript implementation for Node.js
Full and compact format generation

Server-Side Diagnostic Responses:

Compact response generation
Field interpolation
Error versus warning patterns
Mixed response handling

Client-Side Catalog Loading:

TypeScript/JavaScript implementation
Python implementation
Browser caching strategies (localStorage)
Catalog versioning and updates

Client-Side Expansion:

Expansion algorithm
Field substitution
Unknown compact ID handling
Multiple diagnostic expansion

Testing:

Catalog validation tests
Expansion tests
Integration tests
End-to-end examples

5.2 Server Example (Rust)#

Rust
use serde_json::json;

fn generate_compact_response(
    compact_id: &str,
    fields: serde_json::Value
) -> serde_json::Value {
    json!({
        compact_id: {
            "f": fields
        }
    })
}

// Usage
let response = generate_compact_response(
    "xY9Kp",
    json!({"timestamp": "2024-01-15T10:30:00Z"})
);

5.3 Client Example (TypeScript)#

Typescript
class WDPCatalog {
  private catalog: Catalog;

  async load(url: string): Promise<void> {
    const response = await fetch(url);
    this.catalog = await response.json();
  }

  expand(response: object): ExpandedDiagnostic[] {
    const results: ExpandedDiagnostic[] = [];
    
    for (const [compactId, data] of Object.entries(response)) {
      const entry = this.catalog.diags[compactId];
      if (!entry) continue;
      
      let message = entry.message;
      for (const [key, value] of Object.entries(data.f || {})) {
        message = message.replace(`{{${key}}}`, String(value));
      }
      
      results.push({
        code: entry.code,
        compact_id: compactId,
        severity: entry.severity,
        message: message,
        description: entry.description,
        hints: entry.hints
      });
    }
    
    return results;
  }
}

6. Component Interaction#

6.1 Relationship Diagram#

┌──────────────────────────────────────────────────────────┐
│                     Part 9a                              │
│              CATALOG FORMAT                              │
│                                                          │
│  Defines JSON structure:                                 │
│  • Full/compact/minimal formats                          │
│  • Field specifications                                  │
│  • Validation rules                                      │
│  • Distribution strategies                               │
└──────────────────────────────────────────────────────────┘
                         │ produces
                         ▼
┌──────────────────────────────────────────────────────────┐
│                   catalog.json                           │
│                                                          │
│  The catalog file distributed to clients                 │
└──────────────────────────────────────────────────────────┘
                         │ used by
                         ▼
┌──────────────────────────────────────────────────────────┐
│                     Part 9b                              │
│               WIRE PROTOCOL                              │
│                                                          │
│  Defines transmission format:                            │
│  • Compact response structure                            │
│  • Field interpolation                                   │
│  • HTTP patterns                                         │
│  • Expansion algorithm                                   │
└──────────────────────────────────────────────────────────┘
                         │ implemented using
                         ▼
┌──────────────────────────────────────────────────────────┐
│                     Part 9c                              │
│            IMPLEMENTATION GUIDE                          │
│                                                          │
│  Provides code examples:                                 │
│  • Catalog generation (Rust, Python, TypeScript)         │
│  • Server-side response generation                       │
│  • Client-side loading and caching                       │
│  • Expansion logic with examples                         │
│  • Testing patterns                                      │
└──────────────────────────────────────────────────────────┘

6.2 Design Philosophy#

Part 9a specifies WHAT (the catalog structure)
Part 9b specifies HOW (the transmission protocol)
Part 9c provides IMPLEMENTATION (practical code examples)

All three specifications work in conjunction to provide a complete catalog system.

7. Quick Start Guide#

7.1 Catalog Generation (Part 9a + 9c)#

# Build time
$ wdp-catalog-gen --input error-registry.json --output catalog.json

7.2 Catalog Distribution (Part 9a)#

# Upload to CDN
$ aws s3 cp catalog.json s3://cdn.example.com/wdp/catalog-v1.0.0.json

7.3 Client Catalog Download (Part 9c)#

const catalog = new WDPCatalog();
await catalog.load('https://cdn.example.com/wdp/catalog-v1.0.0.json');

7.4 Server Compact Response (Part 9b)#

// Server
let response = json!({
    "xY9Kp": {
        "f": {
            "timestamp": "2024-01-15T10:30:00Z"
        }
    }
});

7.5 Client Expansion (Part 9c)#

// Client
const diagnostics = catalog.expand(response);
console.log(diagnostics[0].message);
// Output: "Token expired at 2024-01-15T10:30:00Z"

8. Applicability#

8.1 Catalog Type Selection#

Single-Namespace Catalogs are applicable when:

Client communicates with a single backend or service
Mobile application with single API
Web application (SPA) with single backend
Individual microservice serving its own diagnostics
IoT device (typical case — communicates with single service)
Single library or SDK

Aggregated Catalogs are applicable when:

Monitoring dashboard displaying multiple services
Edge gateway aggregating multiple devices or services
Log aggregator collecting from multiple sources
SDK bundle with multiple dependencies
Operations and observability tools
Multi-tenant platforms
API gateway or BFF routing to multiple backends

8.2 Catalog System Applicability#

The catalog system is applicable when:

Bandwidth is constrained:

IoT devices with cellular connections
Mobile applications on metered networks
High-volume APIs with cost considerations
International users with slow connections

Client-side expansion provides benefit:

Multi-language support required (localized catalogs)
Cross-system diagnostic resolution (cached catalogs from multiple services)
Client-side error handling and formatting
Rich client-side error UI

Scale is a consideration:

Many diagnostics across distributed systems
Frequent diagnostic transmission
Multiple services generating diagnostics

8.3 Alternative Approaches#

Consider alternative approaches when:

Use case is simple:

Internal tools with fast networks
Low diagnostic volume
Single-language only
Single service (no cross-system resolution needed)

Full responses are already in use:

REST APIs already transmitting full JSON
GraphQL with detailed errors
WebSocket with verbose messages

Team capacity is limited:

Small team without capacity for catalog infrastructure
MVP or prototype phase
Simple monolithic application

Appendix A: References#

A.1 Normative References#

Reference	Title
[RFC2119]	Key words for use in RFCs to Indicate Requirement Levels
[RFC8174]	Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words
[RFC8259]	The JavaScript Object Notation (JSON) Data Interchange Format

A.2 Informative References#

Reference	Title
[STRUCTURE.md]	WDP Specification Structure
[5-COMPACT-IDS.md]	Compact ID Specification
[7-NAMESPACES.md]	Namespace Specification
[9a-CATALOG-FORMAT.md]	Catalog Format Specification
[9b-WIRE-PROTOCOL.md]	Wire Protocol Specification
[9c-IMPLEMENTATION.md]	Implementation Guide