Avro Schema Evolution: Understanding Forward, Backward, and Full Compatibility

When working with event-driven architectures and streaming platforms like Kafka, schema evolution is crucial for maintaining system stability while allowing your data models to grow. Let's explore how Avro schema compatibility works using a real-world example from a sales order system.

The Challenge: Evolving Return Status Enums

Consider this Avro schema field that tracks return order statuses:

{
  "doc": "Status code about this line in the Sales Order.",
  "name": "return_status",
  "type": [
    "null",
    {
      "type": "enum",
      "name": "LineReturnStatus",
      "symbols": [
        "NEW",
        "EXPECTED",
        "REGISTERED",
        "QUARANTINE",
        "RECEIVED",
        "INVOICED",
        "CANCELLED"
      ]
    }
  ],
  "default": null,
  "field_type": "other"
}

As business requirements evolve, you might need to add new return statuses. How do you do this safely without breaking existing consumers or producers?

Understanding Compatibility Types

1. Forward Compatibility

Definition: Old consumers can read data written by new producers.

Why it matters: When you deploy a new producer with an updated schema, existing consumers shouldn't break.

Example Problem:

// OLD SCHEMA (deployed consumers)
{
  "name": "LineReturnStatus",
  "type": "enum",
  "symbols": ["NEW", "EXPECTED", "REGISTERED"]
}

// NEW SCHEMA (new producer adds statuses)
{
  "name": "LineReturnStatus",
  "type": "enum",
  "symbols": ["NEW", "EXPECTED", "REGISTERED", "QUARANTINE", "RECEIVED"],
  "default": "NEW"  // ✅ Critical for forward compatibility
}

What happens:

New producer writes "QUARANTINE" (unknown to old consumer)
Old consumer encounters unknown symbol → defaults to "NEW"
✅ System continues working

Without the default:

Old consumer encounters "QUARANTINE" → ❌ Deserialization fails

2. Backward Compatibility

Definition: New consumers can read data written by old producers.

Why it matters: When you deploy updated consumers, they must handle old data still in Kafka topics or databases.

Example Problem:

// OLD SCHEMA (data already in Kafka)
{
  "symbols": ["NEW", "EXPECTED", "REGISTERED", "QUARANTINE"]
}

// NEW SCHEMA (attempting to remove a symbol)
{
  "symbols": ["NEW", "EXPECTED", "REGISTERED"]  // ❌ Removed "QUARANTINE"
}

What happens:

Old data contains "QUARANTINE"
New consumer cannot deserialize → ❌ System breaks

Golden Rule: Never remove enum symbols if you need backward compatibility.

Safe changes:

✅ Add new enum symbols
✅ Add new optional fields with defaults
❌ Remove enum symbols
❌ Remove fields
❌ Change field types

3. Full (Transitive) Compatibility

Definition: All schema versions can interoperate - both forward AND backward compatible.

Why it matters: In distributed systems, different services may be at different schema versions simultaneously. Full compatibility ensures they all work together.

Evolution Chain Example:

// VERSION 1 (Initial release)
{
  "name": "LineReturnStatus",
  "type": "enum",
  "symbols": ["NEW", "EXPECTED"],
  "default": "NEW"
}

// VERSION 2 (Adds registration tracking)
{
  "name": "LineReturnStatus",
  "type": "enum",
  "symbols": ["NEW", "EXPECTED", "REGISTERED"],
  "default": "NEW"  // Maintains forward compatibility
}

// VERSION 3 (Adds quality control stages)
{
  "name": "LineReturnStatus",
  "type": "enum",
  "symbols": ["NEW", "EXPECTED", "REGISTERED", "QUARANTINE", "RECEIVED"],
  "default": "NEW"  // Maintains forward compatibility
}

Transitive Guarantee Matrix:

Reader ↓ / Writer →	V1	V2	V3
V1	✅	✅	✅
V2	✅	✅	✅
V3	✅	✅	✅

All combinations work bidirectionally!

Fixing the Current Schema

The original schema lacks forward compatibility because it's missing the enum default:

// ❌ CURRENT (Breaks forward compatibility)
{
  "type": "enum",
  "name": "LineReturnStatus",
  "symbols": ["NEW", "EXPECTED", "REGISTERED", "QUARANTINE", "RECEIVED", "INVOICED", "CANCELLED"]
  // Missing: "default"
}

// ✅ IMPROVED (Full compatibility)
{
  "type": "enum",
  "name": "LineReturnStatus",
  "symbols": ["NEW", "EXPECTED", "REGISTERED", "QUARANTINE", "RECEIVED", "INVOICED", "CANCELLED"],
  "default": "NEW"  // Enables forward compatibility
}

Best Practices for Schema Evolution

✅ DO:

Always add enum defaults for forward compatibility
Only add new symbols, never remove them
Make new fields optional with sensible defaults
Use union types with null for optional fields
Document your changes thoroughly
Test schema compatibility before deployment

❌ DON'T:

Remove enum symbols (breaks backward compatibility)
Remove fields (breaks backward compatibility)
Change field types (breaks both directions)
Rename fields without using aliases
Change union order for non-null types
Skip compatibility checks in your CI/CD pipeline

Real-World Impact

In a sales order system processing thousands of messages per second:

Without proper compatibility:

Deploy new producer → Old consumers crash
Lost sales orders
Customer complaints
Emergency rollback
Downtime costs

With full compatibility:

Gradual rollout possible
Old and new versions coexist
Zero downtime
Confident deployments
Happy customers

Implementation Checklist

When evolving your Avro schemas:

☐ Added enum default value
☐ Only added new symbols (no removals)
☐ New fields are optional with defaults
☐ Tested with Schema Registry compatibility check
☐ Documented breaking vs. non-breaking changes
☐ Updated consumer/producer code to handle new symbols
☐ Planned gradual rollout strategy
☐ Prepared rollback plan

Conclusion

Schema evolution is not optional in modern distributed systems—it's inevitable. Understanding forward, backward, and full compatibility ensures your system can evolve without breaking. For enum fields like return_status, always include a default value and never remove symbols.

Remember: The key to safe schema evolution is planning for coexistence, not just compatibility.

Additional Resources

Have you dealt with schema evolution challenges? Share your experiences in the comments below!

{
  "doc": "Status code about this line in the Sales Order.",
  "name": "return_status",
  "type": [
    "null",
    {
      "type": "enum",
      "name": "LineReturnStatus",
      "symbols": [
        "NEW",
        "EXPECTED",
        "REGISTERED",
        "QUARANTINE",
        "RECEIVED",
        "INVOICED",
        "CANCELLED",
        "UNRECOGNIZED"
      ],
      "default": "UNRECOGNIZED"
    }
  ],
  "default": null,
  "field_type": "other"
}

{

  "doc": "Status code about this line in the Sales Order.",
  "name": "return_status",
  "type": [
    "null",
    {
      "type": "enum",
      "name": "LineReturnStatus",
      "symbols": [
        "NEW",
        "EXPECTED",
        "REGISTERED",
        "QUARANTINE",
        "RECEIVED",
        "INVOICED",
        "CANCELLED"
      ]
    }
  ],
  "default": null,
  "field_type": "other"
}

Search This Blog

eVision

Kafka Avro Schema Evolution: Understanding Forward, Backward, and Full Compatibility