4 posts
A catatonic BitMamba 1B — broken tokenizer, 10^18 gradients, a midnight ENOMEM and a CUDA wedge that faked a dead GPU — closed 500 steps and the validation reproduced the eval loss to the fourth decimal. The diary of Phase 4B on a 12 GB RTX 3060.
The codec post ended with a dangling issue: the lm_head diverging in validation. Two weeks later, a methodical audit rescaled the model's head, generated a clean checkpoint, and became the foundation of Phase 4 — the Resurrection.
From the wave field theory paper to BitMamba-2 1B: how a study on wave propagation became the TatuEngine inference engine, and why the Ollama truncation loop (num_ctx 4096 vs 262K) became the Docs First rule that runs across the entire ecosystem.
Out of 78 posts, only 1 was about studies — the Ollama truncation loop. But it was in the studies that wave field theory, Mamba SSM and the BitMamba-2 1B were born. The story of what the blog's most neglected section actually produced.