Radar ·
Pithos: C inference engine for LFM models with offloaded experts on a small GPU
Radar records Pithos, a C inference engine for LFM models that keeps frequently used experts in VRAM and runs the rest on the CPU. The developer reports results on a GTX 1650, above a CUDA baseline.
According to the text archived in Radar, published on 2 November 2026, Pithos is a C inference engine for LFM models. It keeps the most frequently used experts in VRAM, runs the others on the CPU, and reads weights from disk when needed. The developer reports 17.5 to 21.9 tokens per second, against 11.2 to 11.3 tps for a CUDA-based baseline, on a GTX 1650. The available excerpt is truncated, so the full test conditions are not included here.
The technical point is hybrid expert offloading on a small GPU, a relevant trade-off for local inference with limited memory. Rota Nacional does not offer this engine and does not solve this engineering problem. To verify, consult the developer's original publication, reproduce the benchmark on your own hardware and model, and confirm the version tested. If your team uses this material for study, do not paste personal or client data into uncontrolled tools.
Original source: Pithos: motor de inferência em C para modelos LFM da Liquid AI