0xBakeer

Projects

TandemLLM is the main one. The others either fed into it or run beside it.

TandemLLM

TandemLLM began as an inference engine and grew into the environment around it. A request goes in, a router reads what it wants, and the model gets the skills, memory and sandboxed tools that kind of task needs. Underneath, small drafters guess the next tokens and the big model checks the guesses in one pass. A guess it agrees with is a free token. What comes out is the model's own text, only faster. Nothing in the design depends on one model or one machine.

Caches keep the KV and recurrent state of a long conversation, so the next turn starts where the last one stopped. StairCut, the router from the paper, measures what a check costs on the GPU and cuts the draft tree each round to the size that pays. The docs name the one case where the text can differ from plain decoding: a near tie between the two best tokens, where a batched check can round the other way.

The article

  1. TandemLLM and StairCutblog.0xbakeer.com

DeepSeek-V4.1-Flash on one DGX Spark

DeepSeek-V4.1-Flash is 510 GB on disk, and the machine I ran it on has 128 GB of memory. I kept the experts a workload uses in memory, as the checkpoint stores them, and read the rest from the NVMe when a token asked for one.

The series is about what that costs. Held-out loss ranked a broken selection as the better one. A task that needs an expert I left out makes the model keep writing while the text stops making sense. The work then moved to the question under it: which experts stay, and whether how often an expert fires is the right way to rank it.

The series

  1. Part 1Which 139 of 384 experts, and who decidesblog.0xbakeer.com
  2. Part 2Held-out loss said the broken model was betterblog.0xbakeer.com
  3. Part 3Frequency is not contributionblog.0xbakeer.com
  4. Part 4One vote each is not one outcome eachblog.0xbakeer.com
  5. Part 5Ten profiles, one box, and what each one keepsblog.0xbakeer.com
  6. Part 6Long answers find the experts I left outblog.0xbakeer.com
  7. Part 7A 510 GB model on a 128 GB box, measuredblog.0xbakeer.com

Inference Atlas

A public atlas of measured inference setups. Engine, version, quantization, flags and workload together decide a number, so each measurement is one JSON file that records all of them, and the repository is the database.

It shows coverage before speed. A grey cell is a setup nobody has measured yet, and the page tells you how to measure it on your own machine and send it in. A terminal app turns a measured cell into a recipe you can run.

arbiter

Serves typed-decision models such as Laya, on their own or next to a language model. You hand it a text and a list of typed questions, and one forward pass answers all of them with a probability for each. It generates no text, so nothing needs parsing. On one GB10 a single question takes 20.9 ms over HTTP. It runs on NVIDIA GPUs and Apple Silicon and speaks a Jev-compatible API.

The article

  1. My cat woke me at five on a Sunday, so I built a local Jevblog.0xbakeer.com

n-gram

N-gram language models from counting up to Kneser-Ney, in three parts that belong together: an article, a TypeScript implementation with 50 tests on the maths, and a browser explorer that trains on text you paste. Raise n and watch the generated text turn into quotation. Nothing leaves the page.