JSON Event Payloads — Designing Messages for Kafka, RabbitMQ & Event-Driven Systems

REST Payloads vs Event Payloads

Aspect	REST API Payload	Event Payload
Delivery	Synchronous request-response	Asynchronous, fire-and-forget
Audience	Single known consumer	Multiple unknown consumers
Coupling	Tight (caller knows endpoint)	Loose (publisher doesn't know consumers)
Retries	Client retries the HTTP call	Broker retries delivery or routes to DLQ
Versioning	URL path or Accept header	Event type name or schema version field
Size	Can be large (pagination, full resources)	Should be small (reference IDs, not full entities)

Important

Event payloads are not API responses. They describe what happened (past tense: "order.created"), not what a consumer should do. This distinction drives every design decision below.

Event Envelope Design

Event Message Structure

Well-designed event payloadjson

1{
2  "specversion": "1.0",
3  "id": "evt_01HXY9Z2ABC3DEF4",
4  "type": "order.created",
5  "source": "order-service",
6  "time": "2026-04-15T14:30:01.234Z",
7  "datacontenttype": "application/json",
8  "subject": "order-5678",
9  "correlation_id": "req_01HXYZ9ABC",
10  "data": {
11    "order_id": "order-5678",
12    "user_id": "user-1234",
13    "status": "pending",
14    "total": 149.99,
15    "currency": "USD",
16    "items": [
17      { "product_id": "prod-001", "quantity": 2, "price": 49.99 },
18      { "product_id": "prod-002", "quantity": 1, "price": 50.01 }
19    ],
20    "created_at": "2026-04-15T14:30:01.234Z"
21  }
22}

CloudEvents Specification

CloudEvents is a CNCF standard that defines a common envelope for event messages. Using CloudEvents ensures your events are interoperable across brokers (Kafka, RabbitMQ, AWS EventBridge, Google Pub/Sub):

Field	Required	Description	Example
specversion	Yes	CloudEvents spec version	1.0
id	Yes	Unique event identifier	evt_01HXY9Z2ABC
type	Yes	Event type (reverse DNS or dot notation)	com.example.order.created
source	Yes	Event producer URI	/order-service
time	No*	Timestamp (RFC 3339)	2026-04-15T14:30:01Z
datacontenttype	No*	Content type of data	application/json
subject	No	Subject of the event (entity ID)	order-5678
data	No	Event payload	{ "order_id": "..." }

Time and datacontenttype

Although time and datacontenttype are technically optional in the CloudEvents spec, always include them. Every event should have a timestamp, and declaring the content type prevents parsing ambiguity.

Event Naming Conventions

Event type naming patternsjson

1// Pattern: domain.entity.action (past tense)
2"type": "order.created"
3"type": "order.payment.succeeded"
4"type": "order.payment.failed"
5"type": "order.shipped"
6"type": "order.delivered"
7"type": "order.cancelled"
8"type": "order.refund.initiated"
9
10// Pattern: reverse DNS (formal / enterprise)
11"type": "com.example.commerce.order.created"
12"type": "com.example.inventory.stock.depleted"
13
14// Anti-patterns to avoid:
15"type": "createOrder"        // imperative (command, not event)
16"type": "ORDER_CREATED"      // inconsistent casing
17"type": "order-was-created"  // verbose, inconsistent separator

Schema Evolution

Safe vs Breaking Schema Changes

Backward-Compatible Changes

v1 -> v2: adding an optional field (safe)json

1// v1: Original event
2{
3  "type": "order.created",
4  "data": {
5    "order_id": "order-5678",
6    "user_id": "user-1234",
7    "total": 149.99
8  }
9}
10
11// v2: Added optional "discount" field (backward compatible)
12{
13  "type": "order.created",
14  "data": {
15    "order_id": "order-5678",
16    "user_id": "user-1234",
17    "total": 149.99,
18    "discount": 10.00,
19    "coupon_code": "SAVE10"
20  }
21}
22
23// v1 consumers ignore the new fields — no breakage
24// v2 consumers check for the new fields and handle their absence

Handling Breaking Changes

Create a new event type instead of modifying the existing onejson

1// WRONG: Changing "total" from number to object in "order.created"
2// This breaks all existing consumers
3
4// RIGHT: Create a new event type
5{
6  "type": "order.created.v2",
7  "data": {
8    "order_id": "order-5678",
9    "pricing": {
10      "subtotal": 149.99,
11      "discount": 10.00,
12      "tax": 12.60,
13      "total": 152.59
14    }
15  }
16}
17
18// Publish both v1 and v2 during migration period
19// Consumers migrate at their own pace
20// Deprecate v1 after all consumers have migrated

Dead Letter Queues

Dead Letter Queue Flow

Events end up in the DLQ for several reasons:

Failure Type	Cause	Resolution
Parse failure	Invalid JSON (truncated, malformed)	Fix producer serialization, replay from DLQ
Schema violation	Missing required field, wrong type	Update consumer or fix producer schema
Business rejection	Invalid state transition, unknown entity	Investigate data inconsistency, manual fix
Infrastructure	Database down, timeout, network failure	Wait for recovery, automatic retry + replay

Event Payload Anti-Patterns

1. Embedding Full Entities

Anti-pattern vs correct approachjson

1// ANTI-PATTERN: Embedding the full user object (bloated, stale data)
2{
3  "type": "order.created",
4  "data": {
5    "order_id": "order-5678",
6    "user": {
7      "id": "user-1234",
8      "name": "Alice Smith",
9      "email": "[email protected]",
10      "address": { "street": "123 Main St", "city": "..." },
11      "preferences": { ... },
12      "payment_methods": [ ... ]
13    }
14  }
15}
16
17// CORRECT: Reference by ID, let consumers fetch what they need
18{
19  "type": "order.created",
20  "data": {
21    "order_id": "order-5678",
22    "user_id": "user-1234",
23    "total": 149.99
24  }
25}

2. Missing Correlation IDs

Without a correlation_id, tracing an event chain across services is impossible. When a user places an order, the chain might be: order.created → payment.charged → inventory.reserved → shipping.initiated. The correlation_id links all of them.

Real-World Example: E-Commerce Event Stream

Complete order lifecycle eventsjson

1// 1. Order created
2{
3  "specversion": "1.0",
4  "id": "evt_001",
5  "type": "order.created",
6  "source": "order-service",
7  "time": "2026-04-15T14:30:00Z",
8  "correlation_id": "corr_abc123",
9  "subject": "order-5678",
10  "data": {
11    "order_id": "order-5678",
12    "user_id": "user-1234",
13    "items": [{ "product_id": "prod-001", "quantity": 2, "price": 49.99 }],
14    "total": 99.98,
15    "currency": "USD"
16  }
17}
18
19// 2. Payment processed
20{
21  "specversion": "1.0",
22  "id": "evt_002",
23  "type": "order.payment.succeeded",
24  "source": "payment-service",
25  "time": "2026-04-15T14:30:05Z",
26  "correlation_id": "corr_abc123",
27  "subject": "order-5678",
28  "data": {
29    "order_id": "order-5678",
30    "payment_id": "pay_xyz789",
31    "amount": 99.98,
32    "method": "credit_card",
33    "card_last_four": "1234"
34  }
35}
36
37// 3. Inventory reserved
38{
39  "specversion": "1.0",
40  "id": "evt_003",
41  "type": "inventory.reserved",
42  "source": "inventory-service",
43  "time": "2026-04-15T14:30:06Z",
44  "correlation_id": "corr_abc123",
45  "subject": "order-5678",
46  "data": {
47    "order_id": "order-5678",
48    "reservations": [
49      { "product_id": "prod-001", "warehouse": "us-east-1", "quantity": 2 }
50    ]
51  }
52}

Best Practices

✓Use the CloudEvents envelope standard for cross-service interoperability
✓Include correlation_id in every event for tracing event chains
✓Name events in past tense (order.created, not createOrder) — events describe facts
✓Make schema changes additive only: add optional fields, never remove or rename
✓Use a Schema Registry to validate events before they reach consumers
✓Configure dead letter queues for every consumer to prevent data loss
✗Don't embed full entities — use reference IDs and let consumers fetch what they need
✗Don't use event payloads as commands — events describe what happened, not what to do
✗Don't change existing field types — create a new event version instead

Try These Tools

JSON Schema Validator

Validate event payloads against JSON Schema definitions

Open tool

JSON Compare

Diff event schema versions to detect breaking changes

Open tool

JSON Pretty Print & Viewer

Inspect event messages from broker logs

Open tool

Continue Learning

JSON API Responses

Design REST API payloads alongside event payloads

JSON Migration & Versioning

Evolve schemas without breaking consumers

JSON Schema

Validate event payloads with JSON Schema

Real-time JSON

WebSocket and SSE patterns for real-time JSON

Frequently Asked Questions

What is a JSON event payload?

A JSON event payload is a structured message published to a message broker (Kafka, RabbitMQ, SQS) that describes something that happened in a system. It typically contains an envelope (metadata like event type, source, timestamp, and ID) and a data section (the actual event details). Unlike REST request/response payloads, event payloads are consumed asynchronously by one or more subscribers.

What is the CloudEvents specification?

CloudEvents is a CNCF (Cloud Native Computing Foundation) specification that defines a standard envelope for event messages. Required fields include id, source, specversion, type, and time. It ensures interoperability across event systems — a CloudEvents-formatted message from Kafka can be processed by any CloudEvents-compatible consumer regardless of the broker.

How should I version JSON event payloads?

Include a version field in the event envelope (e.g., "datacontenttype": "application/json", "specversion": "1.0"). Use semantic versioning for your event schemas: additive changes (new optional fields) are minor versions, breaking changes (field removals, type changes) require a major version bump and typically a new event type.

What is a dead letter queue and why does it matter for JSON events?

A dead letter queue (DLQ) is a secondary queue where events that cannot be processed are routed instead of being discarded. Common reasons: invalid JSON (parse failure), schema validation failure, business logic rejection. DLQs prevent data loss and let you investigate, fix, and replay failed events.

Should I use JSON or Avro/Protobuf for Kafka messages?

JSON is easier to debug (human-readable) and requires no schema compilation step. Avro and Protobuf produce smaller messages and enforce schemas at serialization time. For high-throughput systems (millions of events/second), Avro or Protobuf can reduce bandwidth and storage by 50-80%. For moderate throughput, JSON with Schema Registry validation is a practical choice.

How do I handle schema evolution for event payloads?

Follow additive-only changes: add new optional fields (backward compatible), never remove or rename existing fields, never change a field type. Consumers should ignore unknown fields. Use a Schema Registry (Confluent or Apicurio) to enforce compatibility checks before allowing schema changes to be deployed.

What fields should every event payload contain?

At minimum: id (unique identifier, UUID recommended), type (event name like "order.created"), source (originating service), time (ISO 8601 timestamp), and data (the event-specific payload). Additional recommended fields: correlation_id (to trace event chains), datacontenttype ("application/json"), and subject (the entity ID the event refers to).