Pure-Rust LLM runtime: one binary serves (OpenAI-compatible), runs local agents, and distills models on their own rollouts — on Apple Silicon and NVIDIA. No Python on the hot path.

Ease Lives Instantly

High-performance CUDA kernels for LLM inference & training — callable from Python, Rust, Go, and TypeScript through one stable C ABI.

A leader agent for attention-heavy work.

The more, the better.

A minimal, streaming-first implementation of an sglang-style text generation stack with clear API / Engine / Backend separation. Benchmark comparisons now contrast streaming mini-sglang against a plain Hugging Face generate() baseline (non-sglang) rather than exposing a non-streaming HTTP mode.