Meet Needle 2: An Open 45M-Parameter Tool-Calling Model That Ships as a 14MB Binary and Runs a Full Session in 28MB of RAM

Cactus Compute has released Needle 2 , an open 45M-parameter model for tool calling, device use, and structured extraction. The entire model ships as a single 14MB binary that runs a full session in about 28MB of RAM. Weights are trained and deployed at CQ2-bit using Cactus Quants, and the model is sealed inside the company’s own C++ engine, so there is no runtime to install and no download at inference time. Reported decode throughput is 500 tokens/sec on a Raspberry Pi 5, 400–1,500 tokens/sec on Meta Quest 3S and Apple Vision Pro, and 300–700 tokens/sec on sub-$200 phones. The design premise is narrow and stated plainly by the team: mapping a messy sentence onto a typed function signature needs no world knowledge and no open-ended prose. That framing is why 45M parameters are enough here, and why the model targets hardware with no GPU and no NPU.

Yes, Needle 2 ships as prebuilt binaries and a static library for macOS, Linux (x86-64, ARM64, ARMv7, RISC-V, MIPS32el), Windows, Android, iOS/watchOS/tvOS, and WebAssembly. Cactus says Pebble already runs Needle locally in the Index 01 app for offline voice actions.

Needle 2 uses what Cactus team calls a Simple Attention Network. The recipe replaces the FFN with a Hadamard MLP, keeps GQA attention, adds engram key-value memory from hashed n-gram tables, and uses multi-lane hyper-connections. The network is 27 layers and 512 wide. The underlying study is on arXiv as A Controlled Study of Attention-Only Transformers .

Pretraining used a proprietary 115B-token corpus, with 38B tokens of post-training. The research team notes LFM2.5-230M was pretrained on 19 trillion tokens.

Needle 2 spends 70 MFLOPs per token, with 35M of 45M parameters matmul-active. LFM2.5 230M spends 460, FunctionGemma 270M spends 540, and Apple FM sits near 6,000.

Weights never decompress into RAM. The 2-bit codes expand inside vector registers and fuse into integer dot products, so the arithmetic path stays int8. One binary probes the CPU at startup and selects a kernel tier: SDOT, NEON, AVX2, RISC-V vectors, wasm SIMD, or scalar.

A byte-level grammar compiled from your JSON schemas constrains every emitted token. Because the matcher knows which tokens are legal before logits exist, the engine skips up to 98% of the vocabulary projection on structural tokens.

Attention uses a 256-token sliding window, and the system turn plus tool declarations are pinned as KV sinks. Memory stays near 28MB regardless of conversation length.