Yandex Open-Sources YaFF: A Zero-Copy Wire Format for Protobuf With Near-Struct Read Speed

Yandex has open-sourced YaFF ( Yet another Flat Format ) under Apache 2.0. It is a high-performance C++ serialization library. YaFF provides a zero-copy wire format for the Protobuf ecosystem. Your .proto file stays the single source of truth. The format only changes how data sits in memory. It concentrates on server-side runtimes.

YaFF is not a replacement for Protobuf. It is an alternative wire format for Protobuf messages. The same .proto schema generates a proto-like C++ API. Reads need no parsing step, so fields come straight from the buffer. Less performance-sensitive code can still parse the wire format back into Protobuf messages. That two-way conversion is what makes module-by-module adoption realistic. You introduce YaFF in one hot path and leave the rest on Protobuf.

Protobuf parsing can consume double-digit percentages of CPU in high-load backends. At scale, that maps to thousands of physical cores. The common zero-copy option is FlatBuffers, also from Google. But FlatBuffers is not a Protobuf drop-in and requires maintaining a separate schema and conversion layer. semantically incompatible with Protobuf. Migrating means duplicated schemas, different schema-evolution rules , and hand-written field converters. Many teams conclude the cost is not worth it. YaFF aims at that gap: zero-copy reads with Protobuf semantics preserved.

A layout decides how a message is stored in the buffer. It changes only the physical representation, leaving the schema and generated interfaces unchanged. YaFF ships four layouts. Fixed is a plain packed struct with no header and a frozen schema. Flat adds a two-byte header and supports schema evolution. Sparse addresses fields through a meta table, fitting sparse schemas. Dynamic is the default and selects Flat or Sparse at runtime. It uses Flat while the schema permits, then switches to Sparse when evolution breaks flat alignment.

Yandex ships a reproducible benchmark suite, built with google/benchmark in a Release build. The numbers below are median nanoseconds per read on an AMD EPYC 7713 with Clang 20.1.8. Lower is faster. In the hot hierarchical case, the Flat Layout reads in 9.79 ns. FlatBuffers needs 37.30 ns, and Protobuf needs 219.35 ns. The raw C++ struct baseline is 8.14 ns. So the Flat Layout reads about 3.8× faster than FlatBuffers here, and about 22× faster than Protobuf. It stays within 1.2× of the raw struct.

Note: The absolute numbers depend on the host CPU and memory. The ratios between formats are expected to hold across hardware.

FlatBuffers and YaFF both read fields by reinterpreting raw memory as the target type. That type-punning leaves TBAA without strong enough facts. So LLVM’s alias analysis falls back to a conservative MayAlias verdict. The compiler then cannot prove that repeated accesses are safe to reuse. Writing root.intermediate().leaf().a() twice re-walks the tree each time. YaFF adds annotations in its generated code that tell the compiler when reuse is safe. YaFF’s generated-code annotations can often help the compiler reuse the access chain, as long as the relevant memory is not modified between reads. As long as nothing writes to memory between reads, YaFF caches the access chain on its own.