[go: up one dir, main page]

Features / Engines

17 engines, written from scratch.

Most LocalAI backends wrap somebody else's engine. These were written from scratch, each one because the thing we needed was a multi-gigabyte Python install, or closed, or nobody had built it yet. What ships instead is a binary and a GGUF file, checked against the reference implementation in CI, running on the machine you already own.

Engines
17
Modalities
7
Python at inference
0
Licence
MIT

Hearing

Turning sound into something a model can act on, words first and then everything else in the room.

C++17

parakeet.cpp

Transcribe a meeting on a laptop CPU and be finished before whisper.cpp has cleared the first minute.

  • About 27x faster than whisper.cpp turbo on CPU, and about 12x on GPU
  • WER 0 against NVIDIA NeMo on every published checkpoint, so the transcript is identical
  • Cache-aware streaming with end-of-utterance detection, for live audio
  • Ten checkpoints, from 110M to 1.1B, and 40 or more locales on the streaming multilingual model
Long-form audio, ours against NeMo on the same machine
parakeet.cpp on GitHub ↗

C++17

moss-transcribe.cpp

Get the transcript, the speaker labels and the timestamps out of a single pass, then export straight to srt or json.

  • 1.58x to 1.78x faster than PyTorch on CPU, on about 1.5x less memory
  • Byte-identical transcript against the reference, cosine 1.0 component by component
  • 3.4 GB at f32 down to 511 MB at q4_k, still byte-identical through q5_k
moss-transcribe.cpp on GitHub ↗

C++17

ced.cpp

Let the model hear a smoke alarm, a dog or breaking glass, not only the words somebody typed.

  • 527 AudioSet sound classes, multi-label, tagged in about 55 ms
  • 6 MB on disk at ced-tiny q8_0, 111 MB at ced-base
  • About 1.25x faster than PyTorch at f32 on half the memory
  • Works over REST and live over the realtime websocket
ced.cpp on GitHub ↗

C++

LocalVQE

Keep a voice loop usable in a real room, with the echo, the noise and the reverb removed before the model ever hears it.

  • Echo cancellation, noise suppression and dereverberation in one pass
  • Streaming and causal, 16 ms latency, 5x realtime on a desktop CPU
  • From a 17 KB linear filter to a 19 MB joint model, pick what your CPU can afford
  • About 21x realtime on a single Raspberry Pi 5 core
LocalVQE on GitHub ↗

The rule we hold them to

We do not ship a port until it matches the original.

Every engine here is gated against the framework it replaces, on the same input, on the same machine. That means a transcript identical to the reference, boxes that land on the same pixels, or a waveform inside a stated tolerance. Speed work comes after that, and the numbers on this page come out of each engine's own benchmark suite.