Cap'n Proto vs Protobuf: why we switched
We'd been using Protobuf for years. It worked. Nobody complained. Then we started hitting the edges: schema migrations that required careful coordination, CPU time spent parsing fields we didn't need, and a serialisation format that couldn't be read without first copying data off the wire. That's when we started looking seriously at Cap'n Proto.
Most comparisons between the two focus on benchmarks. The performance difference is real, but it's not the most interesting part of the story.
What Protobuf Actually Does
Protobuf encodes messages into a compact binary format. When a message arrives, you deserialise it: allocating memory and copying fields into a language-native struct. Only then can you access anything. This is the standard model for most serialisation libraries and it's perfectly fine for the majority of use cases.
What it means in practice:
- Every read requires a full deserialise pass
- Schema evolution is managed via field numbers, removing or reusing them breaks compatibility
- The wire format is opaque without the .proto definition
- Parsing allocates memory proportional to message size
What Cap'n Proto Does Differently
Cap'n Proto's defining feature is zero-copy reads. Messages are laid out in memory exactly as they appear on the wire. When data arrives, you don't deserialise it: you just hand out a pointer to the buffer. Accessing a field is a pointer dereference, not a parse.
This sounds like a micro-optimisation until you're on a hot path processing millions of messages a second. We saw meaningful CPU reduction just from switching, not because our code changed, but because we stopped paying the parsing tax on every read.
A few other things that matter:
- Structs have a fixed layout defined at schema compile time, fields can be added to the end but the existing layout is frozen, making evolution safer and more predictable
- Messages are self-describing via the schema, you can navigate the structure without fully parsing it
- Cap'n Proto also has a built-in RPC system (though we haven't used it seriously)
- The tooling ecosystem is much smaller than Protobuf's
Schema Evolution: Where Protobuf Has a Real Footgun
Protobuf's field-number system is elegant until someone on your team deletes a field and reuses its number for something else. The schema says it's fine; production disagrees when an old client sends the old field type and the new server interprets it as the new one. It's the kind of bug that doesn't show up in tests and surfaces in prod at the worst moment.
Cap'n Proto takes a stricter stance. Fields are positional and permanent. You can add new fields at the end, but you can't repurpose or remove existing ones. More restrictive, but the invariants are clearer and the failure modes are less surprising. We found this easier to reason about across a team where multiple services share the same schema files.
The Honest Tradeoffs
Cap'n Proto is not a drop-in replacement. The generated code is less ergonomic in most languages. The Go support is good but not as mature as protoc-gen-go. The community is orders of magnitude smaller, which means fewer answers on Stack Overflow and more reading the source when something breaks.
Protobuf also has gRPC, which is a genuinely good RPC framework with broad adoption. If you're building public APIs or services that need to interop with anything outside your own infrastructure, Protobuf + gRPC is the pragmatic choice and there's no shame in that.
The Bottom Line
If you control both ends of the wire and you care about parse overhead, Cap'n Proto is worth the migration cost. The zero-copy model is genuinely different and genuinely faster. The stricter schema evolution is a feature once you've been burned by Protobuf field reuse.
If you're building public APIs, need broad language support, or your team is already deep in the Protobuf ecosystem, stay there. The performance gap won't matter for most services, and the ecosystem maturity is real. We switched because our specific constraints made it worth it. Yours might not.