Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M: Bidirectional Encoders That Stay Fast at 8K Context on CPU

Liquid AI has released two open-weight bidirectional encoders, LFM2.5-Encoder-230M and LFM2.5-Encoder-350M . Both are masked language models built on the LFM2 hybrid backbone. Both carry an 8,192-token context.

Encoders sit underneath classifiers, intent routers, safety filters, and PII detectors. Those jobs run continuously, usually without a GPU, and increasingly on longer inputs. BERT established the class. ModernBERT pushed its accuracy, speed, and context. Liquid AI’s argument is that the LFM2 architecture continues that line, because its cost grows more slowly as inputs get longer.

The encoders are not trained from scratch. They are initialized from the LFM2.5-230M and LFM2.5-350M decoder backbones, then converted with three changes:

Architecturally, the backbone interleaves gated short-convolution blocks with grouped-query attention, the same design described in the LFM2 technical report . Both checkpoints use a hidden size of 1024 and a 65,536-token vocabulary, and support 15 languages. The license is the LFM Open License v1.0.

The embed below breaks down the conversion, the full ranking table, the per-task scores, and the CPU latency figures.

Liquid AI evaluated 14 models on 17 tasks pulled from GLUE, SuperGLUE, and multilingual classification. Every model is fully fine-tuned per task, and the reported score is that fine-tuned model’s result.

LFM2.5-Encoder-350M posts a 17-task mean of 81.02 (±1.00), ranking fourth. The three models ahead of it are all larger. They are XLM-R XL at 3.5B (83.06), ModernBERT-large at 395M (81.68), and XLM-R large at 560M (81.34). The top model is nearly 10 times its size.

LFM2.5-Encoder-230M posts 79.29 (±1.02), ranking sixth. It beats ModernBERT-base at 78.19 and every EuroBERT model in the table, including EuroBERT-610M (75.87) and EuroBERT-2.1B (72.19). Both new encoders also score above Liquid AI’s own retrieval siblings, LFM2.5-ColBERT-350M (76.18) and LFM2.5-Embedding-350M (75.68). That gap is the stated reason Liquid AI built a general-purpose encoder instead of reusing the retrievers.

The methodology is the more interesting part, and it is open-sourced under Apache-2.0 . Every model is loaded with fp32 master weights and bf16 autocast, so the table compares models rather than number formats. Every model uses the same AdamW recipe, taken from the EuroBERT card. Learning rate is selected per model and task across 10 rates and 3 seeds. Scores are then reported as the mean over 5 fresh seeds that never touched selection. The transformers version is pinned to 4.56.2 so dependency drift is not an uncontrolled variable.